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i.  ihihodejction 


A.  GENERAL  COMMENTS 

The  Office  of  Management  and  Budget  (OMB)  Circular  li o. 
A-76  requires  federal  agencies  to  inventory  and  review  all 
cf  their  Commercial  Activities  (CA)  .  A  Commercial  Activity 
is  defined  as  cne  which  is  managed  and  operated  either  by 
tne  government  or  by  a  private  firm  and  which  provides  a 
product  or  service  tc  a  federal  agency.  Unless  the  activity 
is  an  inherently  governmental  function  which  is  required  to 
he  retained  in-housef  it  must  be  subjected  to  a  rigorous 
cost  comparison  to  determine  the  most  economical  way  tc 
perform  the  work.  If  the  cost  study  shows  that  it  is  chca per 
for  the  government  tc  perform  the  worn,  in-house  resources 
will  te  used.  However,  if  it  is  mors  economical  for  tn> 
private  sector  to  provide  the  product  or  services,  the  func¬ 
tion  is  contracted  out. 

In  1981,  the  federal  government  spent  532.5  billicr.  or. 
services  cf  a  commercial  nature,  including  maintenance  of 
equipment,  military  base  operations,  and  facility  support 
such  as  housekeeping,  security,  and  food  services. 
[Ref.  1].  Currently,  an  estimated  403,900  federal  employees 
perform  in-hcuse  CA  functions  valued  at  $20  billion.  As 
federal  agencies  move  to  comply  with  A-76  policy,  the  level 
cf  contracting  cut  cf  CA  functions  is  expected  tc  grow 
tremendously.  It  is  imperative  that  this  implem an tit  ion  be 
systematic  and  well  thought  out  in  order  to  achieve  both 
maximum  economy  and  efficiency  of  operations.  [Ref.  2] 


E.  STUDY  OBJECTIVES 


The  principal  objectives  of  this  research  report  »:=•  to 
examine  the  application  of  statistical  quality  control  to  CA 
service  contracts  and  evaluate  improved  designs  for  contract 
administration  organizations. 

C.  SCOEE  AND  ASSUMPTIONS 

Although  the  C cm nercial  Activities  policy  is  applicable 
to  all  federal  agencies,  this  study  will  focus  on  the 
Department  of  Defense  (DCD)  ,  with  particular  emphasis  or.  the 
Navy  in  general  and  the  Naval  Facilities  Engineering  Command 
(NAVFAC)  in  particular.  It  is  assumed  that  the  reader  is 
familiar  with  the  acquisition  process  within  the  Department 
cf  Defense. 

D.  RESEARCH  MET HODOIOGT 

An  initial  literature  search  was  conducted  to  review 
pertinent  instructions,  regulations,  policy  guidance,  indus¬ 
trial  literature,  and  reports  applicable  to  A-76  implementa¬ 
tion.  Interviews  and  discussions  were  conducted  during 
onsite  visits  with  personnel  at  the  following  organizations: 

1.  Naval  Supply  Center,  Oakland,  CA; 

2.  Public  Works  Center,  San  Francisco,  CA ; 

3.  Western  Division,  Naval  Facilities  Engineering 
Command,  San  Eruno,  CA; 

h.  Naval  Weapons  Station,  China  Lake,  CA; 

5.  Public  Works  Center,  San  Diego,  CA; 

6.  Civil  Engineer  Corps  Officer  School,  Pert  Huer.eme, 
CA ; 

7.  McClellan  Air  Force  Base,  Sacramento,  CA; 

8.  Travis  Air  Force  Base,  Fairfield,  CA; 


9.  Public  Works  Department,  Naval  Shipyard  Long  Beach, 
C A ;  and 

10.  Public  Works  Department,  Naval  Postgraduate  School, 
Monterey,  CA. 

Additional  telephonic  con  versations  were  held  with 
personnel  in  the  following  headquarters  organizations: 

1.  Naval  Material  Command,  Wasnington,  D.  C.  ; 

2.  Naval  Facilities  Engineering  Command,  Alexandria,  VA; 

3.  Naval  Supply  Systems  Command,  Washington,  D.C.; 

4.  Office  of  Federal  Procurement  Policy,  Washington, 
D. C.  ;  and 

5.  National  Aeicnautics  and  Space  Administration, 
Washington,  D.  C. 

E.  THESIS  CBG  AN  IZATICN 

Chapter  I  defines  the  Commercial  Activities  policy  and 
presents  the  authors'  objectives  and  methodology.  Chapter  II 
discusses  the  scope,  requirements,  and  implementation  of 
A-76.  Chapter  III  identifies  key  issues  in  CA  conversions, 
while  Chapter  IV  examines  the  application  of  statistical 
quality  ccntrcl  to  CA  contracts.  The  next  two  chapters 
address  the  planning,  budgeting  and  design  cf  the  service 
contract  administration  organization.  Finally,  Chapter  VII 
presents  the  study's  recommendations  for  improvements  in  CA 
contract  conversions. 


II.  COHMEBCIAI  ACTIVITIES  POLICY 


A.  DEVELOPMENT  OF  TEE  POLICY 
1  •  Executive  Branch 

The  government's  policy  for  r.ot  competing  wit h  the 
private  sector  was  first  established  during  President 
Eisenhower's  administration.  In  1955,  the  Bureau  of  the 
Eudget  (EC3)  ,  the  predecessor  of  the  Office  of  Management 
and  Eudget  (OMB),  published  Bulletin  Number  55-4  which 
stared: 


It  is  the  general  policy  of  tha  administration  that  the 
Faderal  Government  will  not  start  or  carry  on  anv 
commercial  activity  to  provide  a  service  or  product  for 
its  own  use  if  sucn  a  product  or  service  can  be  produced 
from  private  enterprise  through  ordinary  business  chan¬ 
nels.  Exceptions  to  this  policy  shall  be  made  by  the 
head  of  agency  only  where  it  is  clearly  demonstrated  in 
each  case  that  it  is  not  in  the  punlic  interest  tc 
procure  such  products  or  services  from  private  enter¬ 
prise.  [Bef.  3] 


In  1966  the  first  Circular  No.  A-76  was  issued  by 
BOB  and  it  represented  a  major  change  in  previous  policy 
statements  concerning  contracting  out.  A-76  reaffirmed  "the 
Government's  general  policy  of  relying  on  the  private  enter¬ 
prise  system  tc  supply  its  needs"  [Bef.  4]  but  if  also 
recognized  some  instances  where  "it  is  in  the  national 
interest  for  tae  Government  to  provide  directly  the  products 
and  services  it  uses."  [Ref.  4] 

The  basic  policy  underwent  another  major  change  with 
the  issuance  of  CMB  Circular  A-76  (revised)  dated  29  March 
1979.  Unlike  the  previous  statements  which  only  stressed 
governmert  reliance  cn  private  enterprise,  the  new  policy 
has  three  guiding  principles.  [Ref.  5] 


a.  Bely  on  the  Private  Sector.  The  Government's  busi¬ 
ness  is  net  tc  be  in  business.  Where  private  sources 
are  available,  they  should  be  looked  to  first  to  provide 
the  commercial  or  industrial  goods  and  services  needed 
by  the  Government  to  act  on  the  public's  behalf. 

b.  Retain  Certain  Governmental  Functions  In-House. 
Certain  functions  are  inherently  governmental  in  nature, 
being  sc  intimately  related  to  the  oublic  interest  as  tc 
mandate  performance  by  Federal  employees. 

c.  Aim  for  Economy;  Cost  Comparisons.  When  private 
performance  is  feasible  and  no  overriding  factors 
require  in-house  performance,  the  American  peocle 
deserve  and  expect  the  most  economical  performance  arid, 
therefore,  rigorous  comparison  of  contract  costs  versus 
m-fccuse  costs  shculd  be  used,  when  appropriate,  to 
decide  how  the  work  will  be  done. 


Tc  support  the  new  emphasis  on  the  economy  of 
government  in-house  performance  compared  to  private 
contractor  performance,  a  Cost  Comparison  Handbook  was 
issued  as  a  supplemert  to  A-76.  One  year  later,  in  April 
1980,  the  Cepartment  of  Defense  (DOD)  issued  its  own  Cost 
Comparison  Handbook  (DOD  4  100.33-H)  to  provide  even  mere 
detailed  instructions  to  DOD  activities.  The  intended 
purpose  cf  these  handbooks  is  to  provide  uniform  procedures 
for  conducting  the  ccst  comparisons  and  to  improve  their 
accuracy  and  validity.  Some  general  ground  rules  include: 
[Ref.  5] 

a)  Beth  the  government  and  commercial  costs  must  be  based 


or.  the  same  scope 

performance. 

cf  work 

and 

the  same 

level  of 

b) 

Standard  cost  factors  will 

b  e 

based  cn 

the  Ccst 

Comparison  Handbook. 

c) 

Full  costs  are  to  be 

used  to 

the 

maximum  ex 

tent  prac- 

tical . 

d)  For  workloads  cf  a  continuing  nature,  prepriced  or 
renewable  options  should  be  requested  from  the  con¬ 
tractor  to  minimize  buy-ins. 

e)  Services  costing  under  $100,000  annually  shculd  be 
contracted  out  without  a  cost  study  unless  the  commer¬ 
cial  price  is  unreasonable. 
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f)  The  ccst  comparison  will  use  a  rate  of  10  percent  per 
annum  as  the  opportunity  ccst  of  investments  and  if 
the  net  proceeds  from  the  potential  sale  of  capital 
assets. 

2.  legislative  Eranch 

As  a  result  of  the  increased  emphasis  on  contracting 
out.  Congress  has  repeatedly  expressed  its  concern  about  the 
implementation  of  the  A-76  policy.  In  fiscal  year  1978  (FY 
78)  ,  Congress  placed  a  one  year  moratorium  on  virtually 
all  contracting  cut  cf  A-76  functions  if  such  a  conversion 
would  displace  a  Government  employee.  [Ref.  6]  At  the  start 
of  FY  83,  Congress  instituted  a  six  month  moratorium  on 
certain  A-76  functions.  Although  the  most  recent  ban 
exempted  studies  on  grounds  maintenance,  refuse  collection, 
food  services,  base  transportation,  laundry  and  custodial 
functions,  it  prohibited  all  other  contracting  out  studies. 
In  addition,  it  prevented  the  conversion  cf  contracts  for 
all  non-exempt  A-76  studies  completed  but  net  yet  awarded. 
[Ref.  7] 

There  are  also  two  Public  Laws  which  guide  COD'S 
decision  to  convert.  The  overall  management  of  DOD 
personnel  resources  is  contained  in  DOD's  Appropriation 
Authorization  Act,  1975,  excerpts  of  which  are  provided  in 
Appendix  A.  This  Act  requires  that  DOD  "use  the  least 
costly  form  of  manpower  that  is  consistent  with  military 
requirements  and  other  needs  of  the  Department  cf  Defense". 
[Ref.  8]  Sections  of  the  DOD  Authorization  Act  of  1981 
providing  further  restrictions  are  also  included  in  Appendix 
A.  The  Act  states  that  functions  cannot  be  contracted  cut 
to  circumvent  any  civilian  personnel  ceiling,  or  unless  the 
Secretary  of  Defense  provides  Congress  specific  notifica¬ 
tion,  certifications,  and  reports  in  a  timely  manner. 
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[Bef.  9]  These  two  previsions  are  considered  to  b*  permanent 
laws  which  will  remain  in  effect  until  changed  by  subsequent 
legislation. 

B.  SCOPE  OF  i-76 

1 •  Exclusions 

The  provisions  of  OMa  Circular  A- 76  dc  net  apply  to 
tne  following  categories  and  situations:  [Bef.  5] 

a)  For  contracting  out  of  personal  services  that  would 
result  in  an  employer-employee  relationship. 

b)  For  major  system  acquisitions  governed  by  QHB  Circular 

A-1C9. 

c)  For  Contractor  Support  Services  which  include 
consulting  services,  studies  and  analysis,  and  profes¬ 
sional  and  management  support  services. 

d)  whenever  implementation  would  be  contrary  tc  law  or 
inconsistent  with  the  terms  of  any  treaty  or  interna¬ 
tional  agreement. 

e)  When  the  activity  is  performed  outside  the  United 
States,  its  territories,  cr  possessions. 

f)  When  products  and  services  are  obtained  from  ether 
federal  agencies  which  are  authorized  cr  required  by 
law  tc  furnish  them. 

g)  In  times  of  declared  war  or  military  mobilization. 

2 •  Inherently  Go ver nme ntal  Functions 

In  addition,  A-76  recognizes  that  inherently  govern¬ 
mental  functions  must  be  performed  in-house.  Thes°  govern¬ 
mental  functions  fall  into  three  main  categories.  The  first 
category  is  the  discretionary  exercise  of  Government 
authority.  This  includes:  [Bef.  5] 
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a)  investigations,  prosecutions  and  other  judicial  func¬ 
tions  ; 

b)  management  of  government  programs  requiring  value 
judgements  such  as  directing  the  national  defense; 

c)  management  and  direction  cf  the  armed  services; 

d)  conducting  of  foreign  relations; 

e)  selection  of  program  priorities; 

f)  direction  cf  federal  employees; 

g)  regulation  of  natural  resources  such  as  the  use  of 
space,  oceans,  and  inland  waterways; 

h)  direction  of  intelligence  and  counter-intelligence 
operations;  and 

i)  regulation  of  industry  and  commerce. 

The  second  category  of  an  inherently  governmental 
function  involves  monetary  transactions  and  entitlements 
such  as  benefit  pregrams;  tax  collection  and  revenue 
disbursements;  control  of  financial  accounts  and  the  admin¬ 
istration  cf  public  trusts.  The  last  type  of  government 
function  is  the  conduct  of  research  and  development  at  test 
facilities  and  the  operation  and  maintenance  support  of 
laoorator ies ,  test  ranges,  test  aircraft  and  ships, 
[fief.  5] 

3 .  commercial  Activities  Subject  to  A- 76 

If  a  service  activity  is  not  specifically  excluded 
from  CMB  A- 76  as  previously  discussed  in  Section  31  of  this 
chapter,  and  is  not  an  inherently  governmental  function  as 
defined  in  Section  B2,  then  it  is  classified  as  a  Commercial 
Activity.  Attachment  A  to  OMB  A-76  provides  approximately 
one  hundred  examples  of  Commercial  Activities  for  fifteen 
different  service  categories.  Such  Commercial  Activities 
may  be  operated  and  managed  either  by  the  government  agency 
or  a  private  commercial  business. 
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4 .  Government  Qcsration  of  a  Commercial  Activity 

In-hcuse  performance  of  Commercial  Activities  car.r.o* 
be  justified  solely  on  the  basis  that  an  activity  supports 
cr  involves  a  classified  program,  or  is  part  of  an  agency's 
basic  mission,  or  that  there  is  a  possibility  of  a  strike  by 
contract  employees.  Government  operation  of  a  Commercial 
Activity  can  only  be  authorized  under  one  of  the  following 
conditions:  [Ref.  5] 

a)  No  Satisfactory  Commercial  Source  Is  Availaoie. 
Government  operation  is  permitted  whenever  it  can  be 
documented  that  either: 

i)  There  is  no  private  commercial  source  capable  of 
providing  the  needed  service,  cr 

ii)  That  the  use  of  a  private  source  would  cause 
unacceptable  delay  or  disruption  of  an  essential 
program.  The  required  documentation  must  b^ 
detailed  in  terms  of  cost,  time,  and  performance 
measures.  The  disruption  must  be  of  a  lasting 
nature  and  not  just  temporary. 

b)  National  Defense. 

i)  Government  operation  by  military  personnel  is 
permitted  whenever: 

1.  The  personnel  are  utilized  in  or  subject  to 
deployment  in  a  direct  combat  or  ccmfcat 
service  support  role. 

2.  The  activity  is  essential  for  exclusively 
military  training. 

3.  The  activity  is  required  to  provide  appro¬ 
priate  work  assignments  fcr  career  progres¬ 
sion  cr  a  rotation  base  fcr  overseas  cr  sea 
to  shore  assignments. 

ii)  Government  operation  of  a  depot  or  intermediate 
level  maintenance  facility  may  be  justified  to 


ensure  a  ready  and  controlled  source  cf 
nical  competence  and  resources  necessary  to 
military  contingencies, 
c)  Lower  Cost.  If  none  of  the  preceeding  conditions  can 
be  met,  government  operation  cf  a  Commercial  Activity 
can  only  be  authorized  when  a  comparative  cost 
analysis,  performed  in  accordance  with  A-76  and  the 
Cost  Comparison  Handbook,  shows  that  in-house  opera¬ 
tion  has  a  lower  total  cost  than  if  it  were  obtained 
from  a  qualified  private  source. 

C.  BEQUIBEHENTS  OF  1-76 

1 •  I n v € nt o r y  cf  Commercial  Activity  Functions 

CHB  Circular  A-76  required  each  agency  to  compile 
and  update  annually  a  complete  inventory  of  all  Commercial 
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M  USB  EH  OF 
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ANNUAL  OPERATING 
COSTS  (BILLION  S) 
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Figure  2.1  DOD  Inventory  of  CA  Functions. 

Activities  subject  tc  its  provisions.  The  initial  DOD 
inventory  was  completed  in  August  1930  and  is  summarized  in 

Figure  2.1.  [  Bef.  10] 
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•  Management  Review 

DCD  Instruction  41  00.  33  requires  chat  a  ccicl-.t^ 
review  cf  all  CA  functions  be  completed  during  FY  60  Through 
FY  84.  Subsequent  reviews  are  required  at  least  cr.ce  every 
five  years.  In  the  event  that  the  circumstances  supporting 
the  initial  approval  are  not  subject  to  change,  then  subse¬ 
quent  reviews  may  be  waived  by  the  Assistant  Secretary  cf 
Defense  (Manpower,  Reserve  Affairs,  and  Logistics)  . 

The  purpose  cf  the  review  is  to  determine  whether 
the  present  method  of  performance  should  be  continued  or  if 
the  function  should  be  designated  for  a  cost  comparison 
analysis  for  possible  change  in  method  of  performance.  In 
making  this  determination,  the  criterion  specified  in  A-76 
and  explained  previously  in  this  chapter  are  used.  If  the 
function  is  presently  contracted,  a  rough  estimate  cf  the 
cost  tc  perform  the  work  in-house  is  prepared.  If  this 
estimated  in-hcuse  cost  is  not  less  than  contract  perfor¬ 
mance  by  10  percent  cf  personnel  cost  and  25  percent  cf  cost 
cf  ownership  of  equipment  and  facilities,  then  the  function 
remains  contracted  out.  However,  if  the  likelihood  exists 
that  in-hcuse  performance  would  meet  the  cost  differential 
criteria,  fcrmal  cost  comparison  analysis  must  be  performed 
tc  determine  the  cheaper  method. 

3 .  Development  cf  Statement  of,  irfor  k 

The  preparation  of  the  Statement  of  Work  (SC*)  is 
one  of  the  most  critical  steps  in  the  effective  implementa¬ 
tion  cf  the  A- 7 6  policy.  Its  design  will  directly  impact 
the  nature  cf  the  solicitations,  the  cost  comparison  process 
and  subsequent  performance  either  by  in-hcuse  personnel  or 
by  contractor  employees.  The  SOU  must  establish  the  govern¬ 
ment's  actual  minimum  requirements  for  performing  the 
service.  These  standards  are  the  ^ame  regardless  cf  w  net  her 
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the  work  is  performed  by  the  government  or  by  the 
contractor. 

The  SOW  constitutes  the  specifications  for  the 
contract.  It  should  be  sufficiently  comprehensive, 
expressing  all  requirements  in  a  clear,  unambiguous  'and 
concise  manner.  It  should  describe  all  duties,  tasks, 
responsibilities ,  and  frequencies  of  perfcrmar.es.  The  sow 
should  be  performance  oriented  and  specify  what  is  to  be 
done  without  prescribing  how  it  must  be  done.  If  specific 
procedures  are  required,  the  government  bears  the  risk  that 
compliance  will  result  in  unacceptable  performance. 
However,  if  the  SOW  establishes  the  minimum  acceptable 
quality  level  (AQL)  ,  then  the  contractor  assumes  full  legal 
liability  fer  meeting  this  standard. 

Along  with  the  SOW,  a  quality  assurance  plan  is 
required  [Bef.  11].  This  plan  sets  the  surveillance 
requirements  and  procedures  for  the  government's  quality 
assurance  evaluators.  The  quality  assurance  plan  helps  to 
ensure  that  adaquate  performance  is  achieved  and  establishes 
the  mechanisms  for  the  administration  cf  the  service. 
Chapters  IV  and  V  will  examine  these  issues  in  greater 
detail. 

4 •  Prepare  and  Audit  I n-House  2s timat e 

The  OOD  Authorization  Act,  1981  requires  that 
government  in-house  estimates  be  based  on  the  "most  effi¬ 
cient  and  cost  effective  organization  for  perfcrmar.ee". 
[Ref.  9].  DOD  Instruction  U1J0.33H  directs  that  : 

[Ref.  12] 

Each  agency  should  ensure  that  Government  ocerations  are 
organized  and  staffed  for  the  most  efficient  perfor¬ 
mance  .  Tc  the  extent  practicable  and  in  accordance  with 
agency  manpower  and  Dersonnel  regulations,  agencies 
sncula  precede  re  views" un der  this  circular  with  internal 
management  reviews  and  reorganizations  for  acco ntDlishing 
the  work  acre  efficiently,  when  feasible. 
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The  activity  is  not  required  to  achieve  this  most  efficient 
organization  (MEO)  prior  to  a  cost  comparison  study  but  it 
must  use  the  MEO  as  the  basis  for  the  government  in-house 
estimate. 

After  the  in-house  estimate  is  prepared,  based  on 
the  established  Statement  cf  Work  and  using  the  most  effi¬ 
cient  organization,  it  is  required  to  be  audited.  The  audit 
is  performed  by  an  independent  audit  agency,  normally  the  \ 

Naval  Area  Audit  Service,  and  must  be  started  120  days  prior 
to  bid  opening.  [Bef.  13]  In  addition  to  ensuring  proper 
compliance  with  A-7€  and  the  Cost  Comparison  Handbook, the 
Audit  Service  reviews  and  approves  the  proposed  SOW  and  the 
3EO. 

5*  Solicitation  and  Evaluation  of  Contractor  Proposals 

Cnee  the  in- house  estimate  has  been  approved,  firm 
bids  cr  proposals  will  be  solicited.  Although  competitive 
negotiations  on  a  firm  fixed  price  basis  are  the  preferred 
method  of  contracting,  formal  advertising  and  other  pricing 
arrangements  may  be  approved  in  rare  instances.  [Bef.  14] 

After  the  contracting  officer  opens  the  bids  or  completes 
negotiations,  he  determines  the  lowest  acceptable  contract 
price  cf  the  responsive  and  responsible  bidders. 

6 •  Cost  Comoaris.cn 

If  the  lowest  acceptable  contract  price  exceeds  the 
total  ir.-heuse  cost,  then  the  performance  by  the  government 
is  assumed  *:o  be  cheaper  and  the  cost  comparison  process  is 
complete.  When  the  contract  price  is  less  than  the  in-house 
estimate,  further  adjustments  are  required  to  evaluate  the 
impact  cf  the  two  alternatives  on  the  overall  cost  of 
government  operations. 
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There  a:;  *.wc  ad  jus  tments  made  to  the  in -he  as  e  Did. 
First,  the  cost  of  capital  for  assets  that  will  be  used  for 
in-house  performance  must  be  added.  Second,  any  ore  time 
cost  associated  with  a  new  start  where  the  function  is  rot 
presently  performed  in-heuse  must  also  be  included. 

The  contract  price  must  also  be  adjusted  for  several 
factors.  A  cost  of  capital  charge  may  be  added  if  govern¬ 
ment  assets  are  required  tc  assure  contractor  performance. 
Conversion  costs  are  added  to  reflect  the  one  time  costs 
incurred  by  the  government  in  shifting  operations  from 
in-house  to  contract.  When  contracting  out  would  result  in 
a  reduction  cf  the  present  level  of  capacity,  the  additional 
amount  of  overhead  which  must  be  absorbed  by  the  remaining 
activities  is  added  to  the  contract  cost.  Finally,  the 
potential  federal  income  tax  revenue  which  would  be  paid  by 
the  contractor  is  deducted  from  his  contract  price. 

After  all  adjustments  have  been  made,  an  existing 
in-heusa  function  is  not  converted  unless  the  projected 
contract  price  is  lcwer  than  the  government  estimate  ty  at 
least  10  percent  of  the  in-house  personnel  related  cost. 
This  differential  is  included  to  account  for  the  possible 
loss  cf  production,  the  temporary  decrease  in  efficiency  and 
effectiveness,  and  ether  unpredictable  risks  that  result 
from  contract  conversion.  {Bef.  12] 

7.  Cecision  and  App eal  Process 

Upon  completion  of  the  cost  comparison,  a  recom¬ 
mendation  is  mads  to  either  award  the  contract  or  to  perform 
the  function  in-house.  The  recommendation,  along  with  the 
cost  comparison  forms  are  forwarded  to  the  approving 
authority  for  review  and  approval.  Once  approved,  the 
results  cf  the  cost  study  are  announced  and  the  detailed 
analysis  is  made  available  to  the  public.  If  no  significant 
discrepancies  are  identified  or  an  appeal  lodged  within  5 
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working  days  (which  may  be  extended  by  the  contracting 
officer  up  to  15  days  for  complex  decisions) ,  the 
contracting  officer  will  either  award  the  contract  or  cancel 
the  solicitation.  In  the  event  the  function  is  to  oe 
performed  in-house,  implementation  of  the  iiEO  must  ce  initi¬ 
ated  withir.  30  days  and  be  completed  within  one  year. 
[Ref.  15] 


D.  IEELIHENTATION  OF  A- 76 

Currently  an  estimated  400, ICO  federal  government 


employee 

s  perform 

commercial  activities 

valued  at 

320 

billion 

anr.ua  11  y. 

Cf  this  amount,  only 

2 6  billion 

ar  e 

eligible 

for  cost 

studies;  the  other  314  bi 

llion  worth 

ar  e 

exempt  from  A-76  for  reasons  of  national  defense.  Although 
progress  is  accelerating  rapidly,  to  date,  only  a  small 
portion  cf  the  eligible  functions  have  received  a  cost 

comparison.  I  he  Office  of  Federal  Procurement  Policy  (CF  F?) 
estimates  that  a  savings  of  over  55  billion  could  be 

achieved  ever  the  next  five  years  if  these  cost  studies  were 
completed.  [Ref.  2] 

Since  1579,  DOD  has  saved  approximately  5140  million  per 
year  as  a  result  of  CA  studies.  In  addition,  an  average  of 
4, COO  personnel  billets  have  been  converted  to  contract  in 
each  cf  the  last  feur  years.  [Ref.  2]  Data  compiled  in 
January  1982,  showed  that  60  percent  cf  the  functions 
reviewed  shifted  to  contract  and  the  average  costs  irepped 
19  percent.  These  reductions  were  widely  distributed 
however,  with  two-fifths  shewing  greater  than  30  percent 
savings,  another  two-fifths  having  savings  between  11  and  25 

percent,  while  the  remainder  saved  10  percent  or  less. 

[Ref.  1] 


In  FY  82,  the  Navy  had  its  best  year  to  date  ir.  imple¬ 
menting  the  A-76  program.  A  total  of  252  cost  studies  were 
completed  ;  more  than  twice  the  number  for  the  previous 
three  year  period.  Cf  the  5,487  civilian  man-years  studied, 
2,060  were  contracted  out  resulting  in  a  savings  to  tne 
government  exceeding  315.9  million.  The  remaining  functions 
that  were  retained  in-hcuse  netted  a  cost  avoidance 
exceeding  317  million.  £Ref.  16] 

As  or  1  Narch  1983  ,  the  Navy  has  identified  5  3,  457 
personnel  billets  as  candidates  for  CA  studies.  The  total 
percentage  under  NAVFAC's  functionai/ccr.tr actual  responsi¬ 
bility  was  54  percent.  Although  NAVFAC  still  has  the  highest 
percentage  in  the  Navy,  its  share  has  decreased  frcm  FY  82 
when  it  was  as  high  as  70  percent.  NAVFAC  took  an  early 
lead  in  CA  contracting  for  the  Navy  oy  providing  standard¬ 
ized  Performance  Work  Statements  to  its  field  activities. 
Cf  all  the  NAVFAC  functions  studied,  the  four  areas  of 
transportation,  grounds  care,  building  maintenance,  and 
janitorial  services  comprise  82  percent  of  the  total. 
[Bef.  17] 
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III.  CONTRACT  CONVERSION  ISSUES 

A.  PACKAGING  OP  CA  FUNCTIONS 

In  implementing  the  A-76  policy  there  are  several  impor¬ 
tant  contract  conversion  issues  which  muse  be  addressed.  One 
issue  concerns  the  different  methods  in  which  Commercial 
Activities  functions  can  be  reviewed  and  cost  compared.  For 
example,  a  single  function  from  the  activity's  inventory  , 
such  as  bus  services,  can  be  studied  individually. 
Alternatively,  several  functions  may  be  combined  unto  a 
package  such  as  vehicle  operations  and  maintenance.  When 
most,  if  r.ct  all,  functions  are  consolidated  by  an  activity, 
a  total  Ease  Operating  Support  (BOS)  package  is  formed. 

A  multi-function  approach  offers  many  advantages  over  a 
single  function  one.  It  facilitates  implementation  of  a-76 
because  it  reduces  the  number  of  cost  comparisons.  As  a 
result,  it  is  a  very  appealing  option  to  those  commands  wno 
are  under  intense  pressure  to  quickly  contract  cut.  Recent 
Navy  experience  has  shown  that  when  smaller  functions  are 
cost  compared  under  A-76  on  a  single  function  basis,  they 
are  extremely  likely  to  remain  in-house.  However,  when 
these  same  functions  are  consolidated  into  a  multi-function 
package  they  are  far  more  iikeiy  to  be  contracted  cut. 
[Ref.  16] 

The  nature  cf  service  contracting  is  such  that  manna! 
labor  alcne  is  insufficient  tc  ensure  satisfactory  perfor¬ 
mance.  What  is  needed  is  an  effective  management  organiza¬ 
tion  that  can  get  the  job  dene  properly.  A  multi-f  u  r.ct  ion 
contract  increases  this  requirement  and  because  of  its 
greater  dollar  value,  attracts  larger  firms  that  have 
increased  management  expertise.  If  the  functions  are 


complex  cc  time  constrained  it  may  require  the  vast 
resources  of  the  larger  firm.  Consolidation  of  functions 
can  also  lead  tc  greater  economies  of  scale  and  more  effi¬ 
cient  use  of  personnel  and  material. 

Tbs  entire  process  of  contract  administration  is  much 
easier  when  dealing  with  a  single  large  contractor  rather 
than  many  small  ones.  The  command  does  net  have  to  contend 
with  many  differing  company  policies,  procedures,  and 
personnel.  There  is  a  higher  procabiiity  for  satisfactcry 
rework  because  whenever  a  problem  is  identified,  only  a 
single  point  of  contact  need  be  reached.  In  addition,  there 
are  fewer  opportunities  for  contractor  tc  contractor 
finger- pointing  with  multi- function  contracts. 

B.  SHALL  BUSINESS  CONSIDER  ATIONS 

Multi-f unct icn  packaging,  while  cf  great  value  in  imple¬ 
menting  CMB  Circular  A-76,  comes  in  conflict  with  another 
national  policy;  the  Small  and  Disadvantaged  Business 
Utilization  (SADBU)  Program.  Public  Law  95-507,  signed  on 
24  October  1978 , provides  the  legal  framework  for  th®  SAC3U 
Frogram.  Section  21  cf  this  law  states;  [Ref.  18] 

It  is  the  policy  cf  the  United  States  that  small  busi¬ 
ness  concerns  and  small  business  concerns  owned  and 
controlled  by  socially  and  economically  disa ivar.ta oed 
individuals,  snail  have  the  maximum  practicable  copcriu- 
nity  tc  participate  in  the  performance  cf  contracts  let 
oy  any  Federal  agency. 

A  small  business  concern  is  one  taat  is  independently 
owned  and  operated,  and  qualifies  under  guidelines  estab¬ 
lished  by  the  Small  Business  Administration  ( SBA)  with 
regard  tc  number  of  employees  and  annual  receipts.  A  small 
disad var.*agsd  business  is  cne  owned  aDd  operated  by  a 
minority  (Black,  Hispanic,  American  Indian,  e-c.).  In  order 
to  ensure  fair  opportunity  to  participate  m  government 
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contracts,  certain  classes  of  procurements  have  keen  set 
aside  for  the  exclusive  participation  of  small  business  or 
have  teen  granted  8  (a)  set  asides  for  small  disadvantage! 
businesses. 

Circular  A-76  requires  that  any  contracts  which  have 
been  awarded  under  authorized  set-aside  programs  will  net  be 
reviewed  fer  possible  in-house  performance.  It  also  directs 
that  functions  previously  performed  in-house  that  are 
suitable  for  a  set  aside  program  ae  awarded  without  a  cost 
comparison.  On  the  ether  hand,  A-76  states  in-house  activi¬ 
ties  in  excess  cf  $  100, 000  annually  will  net  be  set  asiie 
unless  the  conversion  is  justified  by  a  cost  analysis.  This 
last  statement  has  been  interpreted  to  allow  unrestricted 
award  of  multi-f unct icn  contracts  even  though  the  individual 
functions,  if  awarded  separately,  would  require  being  set 
aside  to  small  business. 

On  16  March  1882,  Congressman  Joseph  Addabbo  ( E—  NY) 
expressed  his  concern  to  Secretary  of  the  Navy  John  Lehman, 
that  consolidation  of  base  support  services  under  EOS 
contracts  would  devastate  the  Navy's  sadbu  Program. 
Congressman  Addabbo  claimed  that  many  of  the  functions  being 
consolidated  were  traditionally  performed  by  small  business. 
He  maintained  that  because  of  the  contract's  large  size  and 
complexity,  small  business  could  no  longer  become  prime 
contractors,  regardless  of  their  prior  performance.  He 
further  contended  that  large  primes  will  "usually  chcose  not 
to  subcontract  to  small  or  small  disadvantag ed  firms  for  a 
particular  service  function."  [Bef.  19]  The  Congressman 
concluded  by  requesting  the  Navy  stop  all  consolidations. 

In  a  follow-up  letter  dated  23  April  1982,  Congressman 
Addabbo  chided  Secretary  Lehman  for  not  sending  him  a 
substantive  reply  tc  his  original  request.  He  also  charged 
that  the  Navy  had  accelerated  its  efforts  to  exclude  small 
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and  minority  business 

from 

service  contracts.  As 

svi  2  er.ce. 

he  cited  the  $6.9  Billion 

BOS  contract  awarded 

at  N 

a  va  1 

Weapons  Center,  China 

Lake , 

California  on  18  April 

1982. 

He 

also  identified  six 

othe 

r  BOS  solicitations 

that 

were 

recently  released  by  other  Naval  activities.  In  closing. 
Congressman  Addabbo  renewed  his  request  to  halt  all  further 
consolidations  of  CA  functions.  [Ref.  20] 

In  his  response.  Secretary  Lehman  stated,  [Ref.  21] 

It  is  my  strong  belief  that  consolidated  cor.tr  act  in  a  car. 
be  a  ccst  effective  strategy  often  fosterina  a  'mors 
efficient  use  cf  scarce  acquisition  resources.  'However, 

I  also  believe  that  consolidations  must  be  undertaken 
with  a  keen  awareness  of  the  oojectives  of  tne  small 
business  program. 

Secretary  Lehman  maintained  that  small  business  has  had 
considerable  success  in  capturing  multi-function  contracts. 
In  the  China  Lake  award,  he  explained  that  eigat  of  the 
fifteen  functions,  representing  50  percent  cf  the  contract's 
value,  were  subcontracted  to  small  business.  Finally  the 
Secretary  promised  that  a  significant  portion  of  future 
consolidated  contracts  would  be  awarded  to  SAD3U  firms 
either  in  total  or  in  part  via  subcontracting. 

On  24  May  1982,  Secretary  Lehman  sent  a  memorandum  to 
the  Chief  of  Naval  Operations  (CNO)  outlining  the  Navy's  r.ew 
policy  on  consolidated  service  contracting.  It  makes  three 
major  points:  [Ref.  22] 

a)  Consolidation  cf  existing  small  business  contracts 
shall  only  be  considered  when  there  is  a  reasonable 
expectation  that  it  will  result  in  an  award  to  small 
business.  In  the  event  that  such  an  award  is  not 
accomplished,  individual  function  contracting  must  be 
reinstated . 

b)  Functions  which  are  currently  oeing  performed  under 
8  (a)  contracts  will  not  be  considered  for  consolidation 
unless  consented  tc  by  the  S3A. 
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c)  Contracting  oat  of  current  in-house  CA  functions  shall 
be  done  giving  the  5ADB0  Program  careful  consider at  act 
early  in  the  planning  stages.  In  "-hose  instances 
where  small  business  participation  is  determined  not 
to  be  in  the  best  interest  of  the  Navy,  supporting 
justification  Bust  be  forwarded  to  CNO  (OP  443)  prior 
to  solicitation. 

Cne  week  later,  or.  1  June  1362,  Deputy  Secretary  cf 
Defense  Frank  Carlucci  promulgated  the  DOD  policy  that  is 
still  in  effect  today.  The  policy  directed  that  any  func¬ 
tions  currently  performed  by  small  business  shall  net  be 
considered  for  consolidation.  In  addition,  the  Carlucci  memo 
stated  that  :  [Hef.  23] 

Future  sclic  ita  tiers,  unless  there  are  overriding 
national  security  considerations,  will  be  packaged  sc  as 
net  to  preclude  performance  by  small  and  small  disadvan¬ 
taged  concerns  as  prime  contractors. 

Although  the  issue  appeared  to  be  resolved,  it  has  again 
surfaced  with  the  Cffice  of  Feieral  Procurement  Policy 
proposed  revision  tc  A-76  that  is  currently  being  consid¬ 
ered.  This  proposal  would  direct  tnat  consideration  he  given 
tc  consolidating  CA  functions  into  a  single  statement  of 
work  fer  cost  comparison  and  potential  contract.  while 
admitting  that  consolidation  may  reduce  prime  contract 
opportunities  for  SAEEU  concerns,  it  only  directs  a  reason¬ 
able  balance  be  maintained  between  consolidations  and  single 
function  awards  to  small  business.  (Hef.  15] 

In  addition  to  CFPP's  recent  support  of  multi-function 
packaging,  the  new  Deputy  Secretary  of  Defense,  Paul  Thayer 
has  stated  that  he  intends  to  change  the  DOD  consolidation 
policy  issued  oy  his  predessor,  Frank  Carlucci.  Recognizing 


that  the  currant  policy  all  but  precludes  multi- function  CA 
contracts,  Ihayer  favcrs  an  approach  which  encourages  ccnso- 
lidaticns  geared  to  large  firms  while  simultaneously 
providing  subcontracts  to  SADBU  concerns.  [Ref.  2]  Both  “he 
OFP?  revision  to  A-76  and  the  new  DOD  policy  on  consolida¬ 
tion  are  expected  to  he  promulgated  within  the  next  several 
months  [Ref.  24]. 

Approximately  300C  Navy  CA  functions  involve  fewer  than 
10  civilian  personnel  man-years.  Even  when  consolidated  into 
small  multi-function  contracts,  many  will  still  be  within 
the  capability  of  small  businesses.  Of  the  252  cost  studies 
conducted  in  FY  82,  twenty  were  greater  than  fifty  man  - yc are 
and  cf  these,  only  five  were  over  one  hundred  man-years  of 
effort.  [Bef.  16]  Although  65  percent  of  the  studies  were 
restricted  to  small  business,  these  set  asides  resulted  in 
fewer  contract  awards.  While  75  percent  of  the  civilian 
man-years  in  the  unrestricted  solicitations  were  contracted 
out,  only  37  percent  of  the  small  business  set  asides  were 
awarded.  [fief.  25]  These  results  clearly  demonstrate  that 
the  larger  unrestricted  solicitations  are  mors  competitive 
with  the  government  and  lead  to  greater  conversions  tc 
contract. 

It  would  appear  that  the  Navy  cannot  achieve  the  reason¬ 
able  talance  sought  by  OFPP  unless  it  is  allowed  greater 
latitude  in  consolidating  CA  func-ior.s.  As  additional  func¬ 
tions  are  subjected  to  CA  cost  studies,  an  increasing 
percentage  will  be  Here  complex,  critical,  ar.d  efficient 
since  the  easier  and  less  efficient  functions  will  already 
he  contracted  out.  It  is  therefore  predictable  that  indivi¬ 
dual  functions  and  these  restricted  to  small  business  will 
have  ever  less  success  than  at  present  in  winning  a  contract 
award.  Thus  a  total  reliance  on  single  function  packaging 
will  net  only  be  detrimental  to  small  ousiness  but  will  also 
severely  handicap  the  A-7b  aim  for  economy  and  productivity. 
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C.  QUALITY  ASSURANCE  OF  CA  CONTRACTS 


When  the  federal  government  purchases  items  or  services 
under  a  contract,  it  assumes  the  responsibility  for  ensuring 
that  such  items  or  services  conform  to  stated  contractual 
requirements.  Two  concurrent  processes  commence  at  contract 
award  which  influence  satisfactory  quality. 

The  contractor  establishes  a  quality  control  program 
whereby  management  control  of  materials  or  services  is  exer¬ 
cised  for  the  purpose  of  prevention  of  defects.  Ir.  govern¬ 
ment  contracts,  the  contractor  assumes  the  responsibility 
for  the  execution  of  the  quality  control  process.  [Ref.  26] 

At  the  same  time  ,  a  government  contract  inspection 
crganizaticr.  administers  a  quality  assurance  process. 
Quality  assurance  is  a  planned  and  systematic  approach  of 
observing  service  performance  to  provide  adequate  confidence 
that  the  items  or  services  conform  to  established  technical 
requirements.  The  quality  assurance  process  verifies  the 
required  quality  of  delivered  items  or  services  prior  to 
their  acceptance. 

Under  current  A-76  policy,  two  key  documents  are 


prepared  during  the  pre-so  Iicitation  phase  that  influence 
the  fcllcw-cr.  quality  control  and  assurance  processes.  Thes 
are  known  as  a  statement  of  work  and  quality  assurance  cl 
The  perform  an  ce-oriented  statement  cf  work  describes 
minimum  required  level  of  services  that  will  be  expected 
the  successful  contractor.  The  quality  assurance  plan  sta 
the  procedures  that  will  be  used  to  check  and  ver 
contractor  performance. 

frier  rc  the  1S79  revision  of  A-76,  solicited 
frequently  ^sed  design  specifications  which  establis 
precise  measurements,  tolerances,  or  quality  cent 
requirements  for  the  ccrtractor.  Other  detailed  infermati 
was  provided  when  deemed  necessary.  Tnese  srecificati 


were  primarily  intended  to  obtain  standardization  cf  deliv¬ 
ered  itsas  cr  services.  In  CA  service  contracting,  suer, 
spacificaticns  often  proved  to  be  unwieldy  and  ineffective. 
Standardization  was  net  always  a  suitable  objective  for  the 
service  contract  process  given  a  wide  diversity  in 
contractor  skills  and  the  character  of  operations  at 
different  Navy  activities.  The  government  assumed  substan¬ 
tial  risk  with  this  method  ip.  that  it  guaranteed  acceptable 
results  as  long  as  the  specifications  were  followed. 

On  26  January  1982,  an  OMB  memorandum  directed  that  "-he 
Office  cf  Federal  Erocure ment  Policy  Pamphlet  Nc.  u  be 
designated  as  Supplement  No.  2  to  Circular  A-76.  It  further 
required  that  both  work  statements  and  quality  assurance 
plans  for  CA  functions  be  written  in  accordance  with  this 
new  supplement  .  [Bef.  27]  The  DFPP  Pamphlet  embraces  the 
widespread  utilization  of  performance-oriented  work  state¬ 
ments  instead  cf  detailed,  exacting  design  specifications  . 
The  contractor  is  clearly  and  precisely  told  what  is 
required  but  not  how  he  must  do  it.  This  allows  a  contractor 
more  flexibility  in  performing  the  work.  An  objectively 
defined  end  product  facilitates  the  contractor's  quality 
control  and  the  government's  quality  assurance  effort.  The 
quality  assurance  plan  gives  the  government  inspector  a 
detailed,  written  plan  which  allows  him  to  accurately 
assess  the  contractor's  per  formar.ee. 

The  statement  of  work  design  encumbers  several  review 
steps  and  processes.  First,  functional  areas  subject  tc  cost 
study  must  be  completely  defined.  Ail  resource  inputs,  work 
processes,  and  production  outputs  required  for  successful 
performance  of  the  jcb  function  must  be  identified  and  inte¬ 
grated.  After  this,  a  job  analysis  process  occurs  in  which 
the  structure  of  the  organization  is  designed,  an  itemized 
listing  cf  work  elements  for  the  function  is  enumerated,  and 
standards  of  performance  for  each  work  element  are  set.  In 
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addition,  resources  tc  accomplish  work  input  are  determined, 
performance  indicators  are  listed,  and  deviation  from  accep¬ 
table  standards  are  specified.  [Ref.  28]  when  the  jcb 
analysis  is  complete,  the  essential  rudiments  of  a  contrac¬ 
tual  statement  of  work  will  be  formulated.  Functional  area 
managers  then  consult  with  contract  specialists  and  indus¬ 
trial  engineers  to  complete  the  contract  package. 

The  Air  Force  and  Navy  have  promulgated  regulations 
which  implement  the  precepts  cf  OFPP  No.  4.  Over  the  past 
three  years,  bcth  have  issued  standardized  statements  of 
work  (SOWs)  or  performance  work  statements  (PWS's)  which 
pertain  to  specific  CA  functions  such  as  refuse  collection, 
grounds  maintenance,  or  any  ether  areas  listed  in  A-76 
inventories.  A  field  activity  performing  A-76  cost  studies 
will  utilize  these  PWS's  and  tailor  them  to  incorporate  the 
special  local  requirements.  Approximately  twenty  PWS's  have 
teen  written  for  Navy  Public  Works  commercial  functions. 

Standardized  quality  assurance  plans  have  also  been 
prepared  for  each  statement  cf  work  and  can  be  tailored  by 
local  activities.  These  state  the  methods  that  will  be  used 
in  inspecting  all  contract  requirements.  The  Naval 

Facilities  Engineering  Command  identifies  five  methods  of 
surveillance:  [Ref.  28] 

a)  One  hundred  percent  surveillance  involves  inspection 
of  each  occurence  of  contract  output.  It  is  expensive, 
time  consuming,  and  not  100  percent  reliable. 

b)  Planned  sampling  allows  part  of  the  contract  output  to 
be  evaluated.  It  is  subjective,  and  generally  useful 


d)  Unscheduled  inspections  are  impromptu  spot  checks  of 
the  contractor's  performance. 

e)  Validated  complaints  result  when  customers  express 
dissatisfaction  with  contractor  performance.  Although 
complaints  may  not  be  used  in  lieu  of  the  ether 
methods,  they  can  be  used  to  verify  results  cf  ether 
inspection  methods,  and  make  payment  deductions. 

The  method  cf  inspection  is  tempered  by  the  various 
types  and  frequencies  of  work.  Repetitive,  frequent  perfor¬ 
mance  may  be  best  suited  for  planned  and  random  inspection 
methods  while  infrequent,  critical  work  items  may  require 
100  percent  surveillance  .  Once  a  method  is  chosen,  inspec¬ 
tion  schedules  are  created  for  each  month  of  contract 
performance  .  Evaluation  worksheets  listing  each  work  item 
are  prepared,  inspections  are  conducted,  and  results  are 
recorded.  Good  performance  is  rewarded  while  poor  perfor¬ 
mance  is  required  to  be  corrected. 

When  a  contract  solicitation  is  issued,  the  statement  of 
work  and  ether  mandatory  (boilerplate)  provisions  are  assem¬ 
bled  and  distributed  to  prospective  bidders.  The  Air  Force 
has  adopted  the  practice  of  distributing  its  quality  assu¬ 
rance  plans  (less  actual  inspection  schedules)  to  each 
bidder  in  an  attempt  to  alert  them  cf  the  contract's  quality 
assurance  standards  .  One  key  requirement  of  this  entire 
package  is  clarity,  precise  wording,  lack  cf  ambiguity,  and 
conciseness.  Lefty  and  technical  wording  tends  tc  be 
confusing  and  must  be  avoided. 

The  authors  observed  that  in  several  instances,  more 
attention  is  given  tc  preparation  of  the  statements  of  verk 
than  tc  quality  assurance  plans.  This  is  understandable, 
since  many  DOD  activities  are  rushing  to  comply  with  the 
A-76  requirement  to  complete  cost  studies  by  FY  84.  In  spit° 
cf  this  time  constraint,  it  would  be  counterproductive  for 
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an  activity  to  move  towards  implementation  of  widespread  CA 
service  contracting  without  a  specific  and  definitized  plan 
for  quality  assurance. 

The  next  chapters  discuss  initiatives  in  effective 
surveillance  methods  and  properly  structured  inspection 
organizations.  A  well  designed  quality  assurance  program 
will  result  in  the  optimum  use  of  scarce  inspection 
resources  and  will  help  ensure  adequate  performance  by  CA 
service  contractors. 
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IV.  STATISTICAL  QCALITY  ASSURANCE  TECHNIQUES  FOR  A-76 

CONTRACTS 

A.  I NTRCCUCTION 

During  A-76  cost  studies,  three  documents  are  prepared 
which  Flav  a  key  rcle  in  the  Commercial  Activities  (CA) 
contract  process.  The  first  is  a  contractual  statement  of 
work  which  defines  minimum  requirements  for  services  ordered 
in  the  contract.  The  second,  a  quality  assurance  plan  which 
is  designed  and  integrated  with  the  statement  cf  work, 
summarizes  those  schedules  and  techniques  that  will  he  used 
ty  the  government  tc  verify  contractor  performance.  A  third 
essential  document,  the  cost  estimate,  reflects  the  most 
effective  ard  efficient  government  employee  performance  of 
the  function  being  studied. 

Cr.e  prime  objective  of  A-76  is  cost  effective  mission 
performance  by  federal  agencies.  This  is  demonstrated  by 
comparing  the  cost  estimate  cf  government  employes  perfor¬ 
mance  tc  prices  cf  ered  by  competing  contractors  vhc  parti¬ 
cipate  in  A-76  firm  bid  procedures.  The  low  offerer  gains 
the  right  (usually)  tc  perform  the  function. 

This  emphasis  on  cost  effective  performance  cf  govern¬ 
ment  operations  is  reflected  in  that  guidance  which 
describes  the  preparation  cf  A-76  CA  uality  assurance 
plans.  CCD  publications  describing  the  design  and  iipiemen- 
taticr  cf  quality  assurance  programs  are  : 

a)  Air  Force  Regulation  400-28  of  September  1979; 
t)  Office  of  Federal  Procurement  Policy  Pamphlet  Number  4 
of  October  1980;  and 

c)  Naval  Facilities  Engineering  Command  Manual  MC-327  of 
November  198  2. 
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Each  of  these  publica t ions  calls  for  develop cf 
ccst- ef feet ive  quality  assurance  programs  ty  each  field 
activity  tc  provide  reliable  assessment  of  service  cot *r act 
performance.  Ccst- ef fecti ve  surveillance  does  connote  an 
inspection  system  being  created  at  the  least  possible  ocst; 
acre  acprctn  ately,  the  important  facet  of  such  quality 
assurance  is  to  utilize  existing  cr  new  inspection  systems 
and  resources  in  a  reliable,  dynamic  fashion  to  gain  the 


net  er  mm  at  ion  cr  conr 


oerfcrmance  under 


contracts.  It  snould  have  the  capaoiiity  tc  be  easily  to 
fied  when  CA  contract  workloads  increase. 

The  flaval  Facilities  Engineering  Command  asserts 
importance  cf  properly  designed  quality  assurance  crogra: 
contrasting  old  and  new  inspection  plan  design  philosophic 
[aef.  28] 
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The  ccntractcr  administers  quality  ccr.trcl  over  ? error- 


service  quauitv  is 


tc  procedures  of  top  management  used  tc  determine  t;u*. 
service  activities  are  being  carried  our  in  a  manner  that 
was  established  by  prior  planning  and  goal  setting. 

Quality  control  is  aimed  at  the  prevention  of  unsatis¬ 
factory  performance  of  CA  contract  services.  Contractor 
programs  fccus  on  developing  employee  self  motivation  tc 
render  acceptable  performance  of  service.  Quality  control 
leads  to  increased  profits  for  the  contractor  and  higa 
levels  cf  customer  satisfac tion,  and  facilitates  increased 
employee  productivity  in  work  performance.  (Ref.  29] 

Circular  A-76  emphasizes  effective  contractor  quality 
control  ar.d  government  quality  assurance  by  embracing  a  new 
type  cf  surveillance  procedure  known  as  random  sampling 
inspection.  Traditional  planned  inspection  and  109  percent 
inspection  systems  are  being  replaced  by  these  having  a 
statistical  basis.  Statistical  sampling  techniques  increase 
the  objectivity  of  government  quality  assurance  since  each- 
item  cf  service  has  ar.  equal  chance  cf  being  inspected.  The 
number  of  government  inspectors  required  for  the  surveil¬ 
lance  process  is  reduced,  resulting  in  less  inspection 
costs . 

E.  TEE  EVOLUTION  OF  STATISTICAL  QUALITY  CONTROL 

A  brief  examination  of  the  definition  and  history  of 
atistical  quality  control  will  be  described.  A  definition 
statistical  quality  control  (SQC)  follows:  [Ref.  30] 


atistica*  quality  control  is  tne 
.cal  principles  ana  methods  whicn 
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_  'cnance  cause  variation'  and 

detect  'assignable  cause  variation.'  Variation  due  tc 
hance  causes*  is  inevitable  while  variation  due  tc 
ssiar.able  causes'  can  usually  be  detected  and 
aoDropriate  methods.  Statistical  quality 
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r.trcl  philosophy  is  the  early  detection  of  assignable 
uses  so  that  product „ qu ality _  may  oe  controlled  at  tne 


Assignable  causes  of  variation  are  due  to  differences  in 
machines,  workers,  materials,  and  either  their  ccmpcs it  for. 
or  relationship  to  each  other  over  time  [Hef.  31]. 

SCC  processes  contribute  to  economical  achievements  of 
product  guality  by  applying  a  body  of  theory  dealing  with 
laws  of  large  numbers  and  probabilities  to  various  indus¬ 
trial  and  service  processes.  These  originated  in  1654  when 
Pascal,  a  French  philospher  and  mathematician,  teamed  up 
with  Pierre  Fermat  to  develop  the  t.neory  of  probability. 
[Hef.  30] 

Until  the  1920's,  most  semblances  of  SQC  were  associated 
with  measures  of  central  tendency,  or  averages.  Increasing 
study  at  this  time  was  devoted  to  the  effect  cf  standard 
deviations  cn  control  processes,  and  led  to  the  foundations 
cf  modern  statistical  quality  control.  [Ref.  31] 

Walter  Shew  hart  cf  Bell  Telephone  Laboratories  developed 
the  first  SQC  model,  known  as  the  quality  control  chart.  It 
was  used  to  measure  product  quality  variations  by  both  the 
Bell  system  and  by  Western  Blectric  Company,  and  was 
augmented  by  statistical  sampling  techniques  of  H.  ?.  Dodge 
and  H.  G.  Rcmig.  [Hef.  32] 

Widespread  evolution  cf  SQC  application  techniques 
occurred  in  World  War  II  due  to  the  need  to  minimize  produc¬ 
tion  scrap  losses.  Government  agencies  developed  training 
courses  for  these  personnel  in  industry  who  inspected 
product  or  service  outputs.  After  World  War  II,  American 
industry  further  developed  SQC  tecaniques,  waich  were 
adopted  in  European  and  Asian  countries  as  well.  SQC  became 
an  underlying  basis  for  many  industrial  productivity 
improvements  and  is  still  utilized  extensively,  as  evidenced 
in  modern  Japanese  industries.  [Hef.  32] 

Frevicus  discussion  may  lead  one  to  believe  that  SQC 
techniques  were  designed  primarily  for  manufacturing  appli¬ 
cations,  but  these  are  equally  valid  for  no  r.-maru  f  ac-  ur  mg 
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types  cf  techniques.  Ike  following  instances  illustrate 
representative  applications:  [Hef.  31] 

a)  SCC  was  used  by  the  Census  Bureau  to  control  clerical 
accuracy ; 

b)  Aliens  Inc.,  a  mail  order  business,  utilized  control 
charts  to  establish  accuracy  in  customer  billings; 

c)  Tte  Illinois  Eell  Telephone  Company  used  SQC  techni¬ 
ques  to  assess  clerical  accuracy  in  the  accounting 
department ; 

a)  The  Standard  Register  Company  used  sampling  plans  to 
control  accuracy  of  sales  invoices; 

e)  United  Air  Lines  used  control  charts  to  improve  accu¬ 
racy  in  customer  bookings; 

f)  Statistical  control  techniques  have  been  used  success¬ 
fully  in  the  health  industry  [Hef.  33]  ;  and 

g)  Statistical  quality  control  techniques  have  been  used 
in  highway  and  airport  pavement  construction 

[Hef.  34]. 

Eased  on  the  preceding  observations,  it  is  reasonable  to 
apply  statistical  sampling  techniques  tc  CA  service  contract 
surveillance  programs.  Such  techniques  have  been  utilized 
by  the  Air  Force  in  its  A- 76  contract  conversions. 
Procedures  for  applying  these  techniques  are  stated  in  Air 
Force  Regulation  APR  400-28  . 

This  regulation  incorporates  a  statistical  sampling 
srcdel  known  as  acceptance  sampling  by  attributes.  This  model 
is  more  fully  described  in  Department  of  Defense  Military 
Standard  105D  cf  28  April  1  963  ( MIL-S  TD-105D)  .  The  standard 
has  been  used  successfully  by  defense  industries  since  its 
original  formulation,  and  its  concepts  form  the  basis  for 
worldwide  acceptance  sampling  standards.  MIL-STD- 105D  can 
be  used  tc  inspect  the  following:  [Ref.  35] 

a)  End  items; 

b )  Ccaponents  and  raw  materials; 
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c)  Operations; 

d)  Materials  in  process; 

s)  Supplies  in  storage; 

f)  Maintenance  operations; 

g)  Data  or  records;  and 

h)  Adiinistrative  procedures. 

This  list  is  net  exhaustive.  The  text  of  MIL -STD-  19  5D  is 
presented  in  Appendix  9.  Prerequisite  conditions  for 

successful  use  of  SQC  techniques  are  that  operations  be 
repetitive,  independent,  functional,  ana  affected  by  as 
outside  factors  as  possible  [Ref.  31],  CA  service  contracts 
meet  these  requirements. 

A  review  of  representative  SQC  literature  revealed  the 
following  objectives;  [Ref.  31] 
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process; 

g)  It  sets  and  adjusts  tolerances  and  specification  based 
cn  acquired  production  experience;  and 

h)  It  improves  employee  morale  and  the  tenor  of 
customer-vendor  relation  ships. 

Incorporation  of  SQC  techniques  in  CA  service  contract 
admin  istraticn  may  initially  appear  confusing.  Proper 
training  which  presents  it  in  a  clear,  concise  manner  should 
preclude  this  apprehension  and  lead  to  acceptance  cf  SQC 
techniques  by  both  government  and  contractor  quality  assu¬ 
rance  personnel.  Other  possible  objections  are  that  SQC 


techniques  may  not  te  appropriate  for  certain  business  c: 
product  lines,  and  managers  may  mistakenly  beliva  that  their 
services  would  always  be  performed  in  an  excellent  fashion, 
exclusive  cf  the  use  cf  SQC  methodology. 

Such  objections  might  be  minimized  if  the  advantages  of 
SQC  are  presented.  It  reduces  scrap  and  rework,  increases 
quality  awareness  in  all  employees,  and  enhances  produc¬ 
tivity.  Ibis  leads  to  increased  quantities  of  improved 
products  and  services.  Inspection  oeccmes  more  scientific 
ar.d  reliable  so  that  prediction  of  impending  trouble  car. 
occur.  Inspection  costs  are  reduced,  while  authentic  and 
accurate  records  of  quality  can  be  created. 

C.  LEGALITY  OF  STATISTICAL  QUALITY  ASSURANCE  IN  GOVERNHENT 

PROCUREMENT 

1  •  Introduction 

A  primary  goal  of  government  procurement  is  to 
obtain  timely  and  acceptable  delivery  of  specified  services. 
Contractors  are  alerted  to  this  goal  when  the  government 
includes  mandatory  clauses  which  state  its  rights  in 
conducting  quality  assurance.  A  typical  clause  from  Standard 
Fora  23-A  (Construction  Contracts)  is  as  follows:  [Ref.  36] 

All  work  (which  includes  but  is  not  restricted  to  mater¬ 
ials,  workmanshio,  and  manufacture  and  fabrication  cf 
components)  shall  te  subject  to  msDection  and  test  by 
the  government  at  all  reasonable  times  and  ail  places 
prior  to  acceptance.  Any  such  inspection  and  test  as  for 
the  sole  benefit  of  the  Government  and  shall  not  relieve 
the  contractor  cf  the  responsibility  of  orcviding 
quality  control  measures  tc  assure  that  the  work 
strictly  candies  with  the  contract  requirements.  No 
inspection  or  test  by  the  government  shall  oe  construed 
as  constituting  or  implying  compliance. 

Additionally,  other  special  inspection  clauses  may  amplify 
unique  quality  requirements  cf  a  contract. 
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The  government  can  exercise  great  latitude  ir.  its 
inspection  cf  a  contractor*  s  performance,  conducting  ir  ir.  a 
reasonable  manner  but  increasing  its  intensity  if  signifi¬ 
cant  defects  in  perfcrmance  are  detected.  Unspecified  means 
of  testing  are  allowable  as  long  as  these  are  reasonable  in 
conducting  surveillance.  Methods  that  increase  contractual 
performance  should  net  be  utilized;  concurrently,  government 
surveillance  should  net  interfere  with  contractor  perfcr¬ 
mance.  Inspection  by  means  of  sampling  procedures  has  been 
upheld  in  several  cases  presented  to  the  Armed  Services 
Eoard  of  Contract  Appeals  (ASBCA).  [  2ef .  36] 

In  any  inspection  system,  the  gevernaert  must  avoid 
the  risk  cf  effecting  constructive  changes  to  the  contract 
by  unreasonably  elevating  its  own  surveillance  requirements 
cr  quality  control  requirements  for  the  contractor. 

2.  Sampling  Technique  s  Substantiated  by  Beards  of 
Contract  Appeals 

SCC  techniques  have  been  sustained  by  the  federal 
courts  and  by  federal  agency  3cards  of  Contracts  Appeals. 
In  Vi-Mil  Inc.,  ASECA  16  820,  75-2  BCA ,  para.  11435, 
MIL-STD-105D  was  utilized  to  properly  estimate  quantities  of 
defective  ccats  that  occurred  in  several  production  lots. 
Government  inspectors  properly  organized  random  samples  to 
ascertain  contractor  perfcrmance  and  were  correct  ir. 
concluding  that  sample  results  were  representative  cf  entire 
production  lets. 

In  Goldring  Sacking  Company,  ASBCA  7736  ,  1962  BCA, 
para.  3392  ,  a  government  decision  to  terminate  for  default 
on  the  basis  of  sampling  results  was  sustained  by  the  Beard. 
An  inspector  checked  11  meat  loins  out  114  total  loins  and 
found  defects  in  each  sample. 
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The  government's  choice  of  acceptable  defect  levels 
was  sustained  in  Precision  Products,  AS BCA  14284,  70-2  5CA , 

para.  8447  in  regards  to  sampling  inspection.  It  was  made 
clear  that  such  choices  must  be  based  on  reasonable  govern¬ 
ment  requirements;  if  defects  are  critical  in  nature,  it  is 
allowable  to  state  that  no  defects  or  defective  performance 
will  be  permitted. 

with  no  method  of  inspection  specified  in  the 
contract,  the  government  proceeded  to  use  sampling  techni¬ 
ques  in  assessing  product  characteristics  in  Frank  and 
Warren,  ASBCA  10259,  65-2  BCA,  para.  5102.  The  Beard  found 
that  sample  sizes  utilized  were  sufficient  to  allow  a  termi¬ 
nation  for  default. 

In  Associate  Aircraft  Tool  and  .Manufacturing  Inc., 
ASBCA  7255,  1963  BCA,  the  Board  stated  that: 

where  the  government  purports  to  reject  an  entire  let  of 
items  or.  the  basis  of  an  inspection  less  than  the  full 
quantity  delivered  (the  inspection  sampling)  ,  the 
inspection  sample  must  either  Be  representative  of  the 
entire  lot,  or  in  accordance  with  a  sampling  and  control 
plan  acreea  to  in  the  contract. 

The  beard  also  emphasized  that  inspectors  properly  designate 
the  manner  cf  forming  inspection  lots,  the  determination  of 
sample  sizes,  and  the  manner  in  which  contractors  present 
lots  for  inspection. 

Sampling  procedures  for  inspection  of  manufactured 
products  were  included  in  contract  provisions  for  Metal  Tech 
Inc.,  ASECA  14328,  72-2,  para.  9545.  The  Board  sustained 
the  manner  in  which  delivered  items  were  rejected  and 
sustained  the  government's  termination  of  the  contract, 
citing  the  Frank  and  Warren  decision  which  stated  that  the 
government  is  not  obligated  to  inspect  eaca  defect  in  ail 
delivered  supplies  when  forming  a  basis  for  lot  rejection. 
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In  a  final  case,  American  Quality  Assurance 
Engineering,  ASBCA  No.  114  17,  1  1466,  1  1544,  ar.d  1  1747,  66-1 
ECA,  para.  6986,  the  government  used  a  liquidated  damages 
clause  whicn  snared: 

The  government  will  inspect  approximately  10  percent  of 
the  tasks  performed  daily  by  the  contractor.  Of  these  an 
acceptable  quality  level  of  a  daily  average  of  net  acre 
that  eighteen  tasks  has  been  estaolished.  If  fer  any 
calendar  month  the  contractor  exceeds  this  average  he 
shall  pay  to  the  government  as  liquidated  damages  for 
excess  administrative  costs  the  sum  of  3250. 0!K  The 
daily  average  of  unsatisfactory  performed  tasks  is 
obtained  by  dividing  the  total  of  unsatisf  actoril  y 
performed  tasks  for  the  month  by  the  number  of  working 
days  for  the  month. 

In  this  case,  government  payment  withholdings  were  in 
dispute.  The  government  based  these  on  judgmental  samplings 
(not  random).  Some  withholdings  were  sustained  while  ethers 
were  denied  by  the  Ecard.  Deductions  from  total  monthly 
invoices  based  on  defect  percentages  observed  in  the  sample 
were  found  to  be  correct.  The  Board  disagreed  with  enlarging 
observed  percentages  to  100  percent  of  the  projected  defi¬ 
ciencies,  and  applying  these  to  the  entire  month’s  perfor¬ 
mance;  it  believed  that  this  enlargement  was  conjectural  and 
lacked  sufficient  accuracy.  Utilization  of  Mil  std  1Q5D 
sampling  techniques  might  have  injected  sufficient  accuracy 
into  the  surveillance  process,  leaving  no  room  for  doubts 
about  the  propriety  of  deductions. 

3.  0 AO  Substantiation  of  Statistical  £ualiox  Assurance 

lechniou  es 

A  recent  General  Accounting  Office  (GAO)  ruling 
substantiates  government  inspection  by  statistical  sampling 
methods.  In  Environmental  Aseptic  Services  Administration 
and  Larsen  Euilding  Care,  8-207771  of  28  February  1983,  GAO 
evaluated  protests  against  Air  Force  implementation  of 
extrapolated  deductions  using  MIL-STD-105D.  An  issue  of 
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dispute  was  The  definition  of  work  itams, 


t  i  gr.t 


as  each  one 

consist  cf  several  subsidiary  tasks.  For  example,  cleaning 
cf  rccms  could  involve  sweeping  floors,  emptying  trash  cars, 
dusting  blinds,  and  several  other  tasks.  3ecause  of  the 
deducticr  provisions  cf  an  Air  Force  CA  contract  solicita¬ 
tion,  the  protestors  claimed  that  one  defective  subsidiary 
task  could  lead  to  rejection  cf  the  entire  room  just  as  if 
all  subsidiary  tasks  were  judged  to  be  unsatisfactory. 

The  GAO  agreed,  finding  that  the  government  require¬ 
ments  were  net  fair  or  reasonable  and  could  be  viewed  as  a 
penalty  system.  GAO  recommended  that  the  Air  Force  distin¬ 
guish  between  vital  and  non-vital  tasks,  establishing 
reasonable  deduction  rates  for  both.  This  has  resulted  in 
more  detailed  breakouts  of  work  requirements  for  certain  Air 
Force  CA  contract  so  licitat  ions. 

The  GAO  emphasized  that  enclosing  the  quality  assu¬ 
rance  provisions  in  solicitations  was  clearly  for  the 
benefit  cf  the  government  and  not  potential  offerors.  These 
can  not  be  disputed,  r.or  car.  failure  by  the  government  to 
adhere  tc  them  form  a  basis  for  protest. 

4 .  Summary 

The  preceding  discussion  points  out  a  sample  of 
legal  cases  in  which  government  use  cf  statistical  quality 
assurance  teenni^ues  were  disputed.  These  were  found  tc  be 
valid  and  applicable  to  government  procurement.  These  tech¬ 
niques  must  be  reasonable  and  net  increase  stated  standards 
cf  performance.  Use  of  SQC  techniques  are  the  preroaative  of 
the  government  and  net  the  contractor.  Hsiecticns  or  deduc¬ 
tions  based  on  sampling  must  he  representative  of  the  lets 
cf  work  cr  service  being  performed. 

sgc  techniques  have  a  legal  basis  when  applied  tc  CA 
service  contracting.  Ensuing  discussion  will  examine 
various  statistical  methods  which  are  being  used  or  are 
being  studied  for  their  possible  application. 


0.  AIR  FORCE  Cl  CONTR  ACTING  UNDER  AFR  400-28  AND 


BIL-STD-  105D 

1  •  1st  red  action 

The  Air  Force  first  issued  policy  requiring  the 
concurrent  design  of  performance-oriented  statements  of  work 
and  matching  quality  assurance  plans  for  CA  service 
contracts  in  Air  Force  Regulation  AFR  400-28.  Subsequently, 
standardized  statements  cf  work  and  quality  assurance  plans 
which  could  be  tailored  to  local  command  needs  were  devel¬ 
oped  and  distributed.  AFR  400-28  required  the  use  cf  random 
sampling  procedures  based  on  MIL-STD-  105D,  along  with  ether 
inspection  methods.  Other  executive  agencies  followed  the 
Air  Force  lead.  The  Office  of  Federal  Procurement  Policy 
issued  OFEP  Pamphlet  No.  4  ,  which  embodies  the  procedures 
stated  in  AFR  400-28.  The  Naval  Facilities  Engineering 
Command  issued  its  cwn  maintenance  manual,  NO-327,  which 
calls  fer  performance-oriented  statements  of  work  but  dees 
not  implement  the  use  cf  MIL-STD-105 D.  NAVFAC  is  exploring 
the  use  of  another  statistical  method,  confidence  level 
estimation,  which  is  examined  later  in  tnis  study. 

2.  Military  Standard  1 05D  Concepts  and  Issues 

Eefora  examining  specific  Air  Force  SQC  policy,  it 
is  necessary  to  elaborate  on  the  basic  statistical  concepts 
and  issues  cf  MIL-STE-105D.  These  are  based  on  acceptance 
sampling  and  are  described  in  simple  terms  by  A.  J.  Duncan, 
a  noted  SCO  author:  £Ref.  32] 

A  company  receives  a  shipment  of  goods.  It  samples  the 
shipment  and  either  accepts  it  as  conforming  to  stan¬ 
dards  cr  rejects  if.  If  the  company  rejects  the  lot  as 
being  fcelcw  standard,  it  may  be  returned  to  the  supplier 
or  it  may  be  kept,  aeoenaina  cr.  how  badly  the  aooas  ere 
needed  or  what  a  rranaem  er.ts  have  been  made  with  the 
supplier.  Possibly  there  will  be  a  price  concession  or. 
rejected  lets.  It  is  to  be  emphasized  that  the  purpose 
of  acceptance  sampling  is  to'  determine  a  course"  of 
action,  net  to  estimate  lot  quality.  Acceptance  sampling 
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specifies  a  procedure  -hat,  if  aoplied  to  a  series  of 
lots,  will  give  a  specified  risk' of  accepting  lots  of 
given  quality. 

In  ether  words,  acceDtance  samoiing  yields  quality  assu¬ 
rance.  It  is  also  emphasized  that  acceptance  sampling  is 
not  an  attempt  to  'control*  quality.  The  latter  is  the 
purpose  of  control  charts;  these  guide  the  engineer  in 
modifying  procedures  so  as  to  turn  out  better  products. 

Under  acceptance  sampling,  the  attributes  of  a 
product  are  judged.  Attributes  are  that  property  of  a  unit 
that  classify  it  as  bad  or  good.  Quality  characteristics  of 
a  unit  are  either  within  specified  limits,  or  are  not 
conforming.  [Ref.  11]  Submission  of  good  guality  services 
results  in  high  rates  cf  acceptance,  while  products  of  pcor 
guality  incur  a  high  rate  of  rejection. 

For  CA  contracts,  single  sampling  plans  fer  frac¬ 
tions  defective  are  used  mest  frequently.  These  define  a 
sample  size  that  is  to  be  taken  and  a  number  of  defective 
units  which  can  not  be  exceeded  in  order  to  prevent  lot 
rejection.  [Sef.  32]  As  an  example,  a  sampling  plan  iray 
call  for  a  sample  of  100  CA  service  work  items  to  be  taken 
from  a  monthly  lot  cf  performance.  Two  or  less  defectives 
result  in  let  acceptance,  while  three  or  more  lead  to  lot 
rejection.  Such  stated  constraints  lead  to  the  ccnstr action 
of  an  operating  characteristic  curve,  which  illustrates  how 
the  probability  of  acceptance  of  a  lot  varies  with  the 
guality  cf  the  material  offered  for  inspection.  At  lew  rates 
of  discovered  defects,  the  probability  cf  lot  acceptance 
will  be  high.  At  high  rates  of  discovered  defects,  the  prob¬ 
ability  cf  let  acceptance  will  be  low.  Operating  character¬ 
istic  curve  profiles  can  be  adjusted  by  varying  lot  and 
sample  sizes  or  by  varying  the  acceptable  defect  rates 
(acceptable  quality  level,  AQL)  as  shown  in  Appendix  C.  Such 
operating  characteristic  curves  illustrate  the  protection 
offered  to  both  ccntractcrs  and  government  customers. 
Application  of  military  Standard  1Q5D  requires  the  following 
sequence  cf  planning  activities. 


and 


3 •  Flaming  Activity  S eguence  for  MIL-5TD-105D 
a.  Lot  Sizes  for  MIL-STD-105D  QA  Plans 

In  using  random  sampling  procedures 
!1IL-S1D- 1 35D  guidelines,  the  size  of  work  item  lets  must 
first  be  determined.  Lots  could  be  the  total  number  of  rooms 
in  a  building  that  are  cleaned,  the  number  of  vehicles  that 
undergo  preventative  maintenance  or  the  number  of  refuse 
containers  that  are  serviced  on  a  monthly  basis.  Lots  can  be 
accumulated  cn  other  than  a  monthly  basis,  but  should  be 
repetitive  or  continuous  in  nature.  All  lot  items  should  oe 
homogenous,  or  have  the  same  characteristics. 

t.  Sample  Size  Determinations 

When  a  let  size  is  known,  taoles  in  M IL-STD-1 Q5D 
are  consulted  to  determine  an  appropriate  sample  size  in 
judging  the  characteristics  of  the  lot.  For  one  let  size, 
there  are  three  different  sample  sizes  corresponding  to 
three  levels  of  inspection  intensity.  Level  I  is  utilized 
when  smaller,  or  reduced,  sample  sizes  are  sufficient;  less 
discr  iminat  icr.  is  necessary. 

Level  II  is  the  normal  level  of  inspection 
intensity.  Sample  sizes  derived  from  Level  II  tables  are 
used  most  frequently  for  CA  contracts. 

Level  Ill's  larger  inspection  samples  are  used 
when  more  discrimination  of  product  guality  is  necessary, 
resulting  in  tightened  inspection.  These  are  used  when  there 
are  major  observed  declines  in  product  quality. 

c.  Determination  of  Acceptable  Quality  Levels 

The  next  activity  in  the  sequence  of  designing  a 
quality  assurance  plan  is  the  determination  of  an  Acceptable 
Quality  Level  (AQL).  This  variable  is  described  in 
MIL- STD- 1 05 E  as  follows:  [  Bef.  35] 
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The  designation  of  an  AQL  shall  net 
supplier  cas  the  right  to  Knowingly  supply  any 
unit  of  product. 

The  AQL  tc  be  used  will  be  desianated  in  the  contract  or 
by  tha  responsible  authority. 


applicable. 


Duncan  states  the  following 

AQL:  [Hef.  32] 


in  reference  tc  the 


In  applying  the  J1IL-STD-  105D  it  is  expected  that  in  a 
conference  between  a  supplier  and  a  military  aaency  it 
will  be  made  clear  to  the  supDlier  whate  for  purposes  cf 
acceptance  sampling,  the  agency  consioers  to  be  accep¬ 
table  quality  levels  for  orcauct  characteristics. 


With  a  specified  AQL  and  inspection  level  inten¬ 
sity,  the  sample  size  can  be  determined  by  using  tables 


T 


value,  the  let  is  accepted.  If  the  number  of  defectives  are 
equal  tc  or  greater  than  the  tejSS  1  number .  the  let  is 
rejected . 

d.  Inspection  Schedule  Design  Us_ng  Random  Number 
Ta  b  1  es 


The  next  step  in  the 

guali 

ty  assurance 

plan 

design  is 

the  choice  of  items  that 

will 

be  included 

in  a 

sample  by 

using  randci  number  tables. 

The 

process  begi 

r.s  by 

assigning  eacn  item  ct  work  in  a  certain  functional  category 
(such  as  service  of  heusing  area  trash  containers)  its  own 
unique  inventory  number. 

A  random  number  table  is  then  consulted. 
Ccrrespcnder.ce  between  wcrlc  elements  in  the  lot  and  digits 
listed  in  the  table  is  established  (a  numbering  system).  A 
route  through  the  table  is  selected  and  is  followed  by 
choc  sing  numbers  according  to  this  pattern.  A  starting  point 
is  fixed,  and  the  table  is  used  until  the  required  number  of 
sample  items  is  chosen. 

In  using  random  number  taoles,  the  selection  of 
inspection  sample  items  occurs  such  that  each  has  an  equal 
chance  cf  being  included  in  a  sample.  Detailed,  explicit 
instructions  on  random  number  table  usage  are  provided  in 
A?R  UC0-28,  07PP  No.  4,  and  NA7FAC  MO-327.  The  quality 
assurance  inspector  lists  each  item  or.  a  schedule  and 
conducts  inspections  at  the  appointed  time.  Results  are 
documented  in  writing  to  aid  in  determining  the  accept¬ 
ability  or  rcn-acceptabilit y  cf  contractor  performance. 

a.  The  Disposition  cf  Inspection  Results 

After  inspection  findings  are  complete,  alterna¬ 
tive  courses  of  action  may  be  pursued  by  the  activity 
contract  inspection  division.  When  AQL’s  are  exceeded, 
rv/ment  deductions  cr  lot  rejections  may  oe  made.  If  two  of 
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are 


U- 

►  4 

:« 


fiva  consecutive  lots 
tightened  inspection  is 
sizes.  Rejected  lots  ma 
corrected,  but  this  is 


rejected  (AQL's  are  exceeded;, 
instituted  utilizing  larger  sample 
y  be  resubmitted  after  defects  are 
done  only  at  the  discretion  of  the 


government. 

When  ten  consecutive  lots  suoject  to  tightened 
inspection  ara  rejected,  performance  is  halted.  It  is 
conceivable  that  a  termination  could  ensue  cn  or  before  this 
point.  Thare  is  a  provision  for  exceptional  contractor 
performance.  When  ten  consecutive  lots  subject  tc  normal 
inspection  have  been  accepted,  a  switch  to  reduced  inspec¬ 
tion  (smaller  sample  sizes)  may  be  made. 


f.  Overview 

If  quality  assurance  plan  designs  based  cn  the 
preceding  sequence  cf  activities  are  conducted  properly, 
hlL-STD- 1 G5D  will  provide  for  effective  and  reliable 
surveillance  of  CA  service  contracts. 

The  concept  of  acceptable  quality  levels  must  be 
carefully  understood.  It  is  not  an  aspect  of  specifications 
cr  cf  performance.  It  is  instead  a  notification  to  contrac¬ 
tors  that  government  surveillance  plans  will  allow  r.c  amount 
cf  defects  observed  in  the  sample  to  be  greater  than  the  AQL 
specified  by  tne  government.  [Sef.  32] 

The  most  commonly  used  fable  in  the  standard  is 
the  single  sampling  plan  (Table  II- A)  .  It  is  designed  sc 
that  alcr.c  diagonal  path  in  the  table,  the  product  of  the 
AQL  and  the  sample  size  is  nearly  constant.  This  has 
resulted  in  a  limited  set  of  AQL’s  which  may  be  utilized,  as 
shown  in  Appendix  3. 

Cne  criticism  of  MIL-STD- 1Q5D  is  that  prcbabili- 


probabilities  cf  acceptance  for  any  given  AQL.  This  notion 
was  rejected  since  it  was  believed  that  substantially 
greater  risks  are  pcsed  to  suppliers  of  large  lots  (and 
corresponding  sample  sizes)  when  these  are  submitted  for 
inspection,  than  for  suppliers  of  smaller  lots  and  samples. 
Increased  sample  sizes  actually  protect  both  government  and 
contractor  interests  as  these  allow  more  accurate  assess¬ 
ments  of  sample  and  lot  qualities.  Therefore,  increased 
probability  rates  cf  acceptance  with  increased  lot  and 
sample  sizes  is  logical.  [  Ref.  32] 

4.  Air  Z2.R£§  Implementation  of  HI L-S IS- 1 05  D  in  CA 

Contracts 

In  addition  tc  statistical  sampling  techniques,  the 
Air  Fcrce  employs  other  surveillance  methods.  One  of  these, 
a  management  information  system,  is  utilized  tc  properly 
ascertain  the  contractor's  performance.  Information  supplied 
by  this  method  may  obviate  the  need  tc  install  random 
sampling  inspection  systems.  Such  systems  may  be  reports 
supplied  by  contractors,  or  by  government  customers  who 
receive  CA  contract  services. 

Surveillance  checklists  are  another  method  cf  CA 
surveillance  suggested  in  AFR  400-28.  However,  this  systemis 
not  recommended  if  a  management  informations  system  or  a 
random  sampling  system  can  be  installed,  since  checklists 
are  a  form  cf  planned  sampling  and  may  be  subjective. 

Formal  customer  complaint  systems  provide  supplemen¬ 
tary  information  describing  contractor  performance. 
Customer  complaints,  under  Air  Force  policy,  are  seldom  used 
in  rejecting  services  or  making  payment  deductions.  When 
random  sampling  systems  are  in  effect,  these  cannot  function 
as  substitutes  for  random  observations,  but  may  be  used  as 
supplementary  documentation.  Guidelines  given  in  AFE  4C0-23 
enable  an  activity  tc  properly  set  up  a  customer  complaint 
system. 
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con  tract  value,  entering  these  on  Performance  Require  ire  nts 
Summary  sheets  and  returning  them  with  the  cid  submission 
package.  Such  percentages  are  later  used  in  making  deduc¬ 
tions  for  unacceptable  performance. 

It  should  be  noted  that  the  use  of  MIL-S ID-  105D  may 
allow  ctserved  defects  rates  greater  than  the  specified  AQL 
when  the  contractor  reaches  cr  exceeds  the  reject  number. 
For  example,  a  lot  cf  20  00  items  is  checked  with  normal 
sampling  intensity  (Level  II)  and  an  AQL  of  10  percent.  A 
sample  of  125  items  is  required;  the  let  is  to  te  rejected 
if  22  defects  are  observed  in  the  sample.  If  22  defects  are 
later  discovered,  the  observed  defect  rate  (17.6$)  exceeds 
the  AQL  (10$)  and  rejection  would  occur.  Sven  though  any 
defects  greater  than  13  would  cause  the  observed  defect  rate 
to  exceed  the  AQL  cf  10$,  this  method  allows  up  to  21 


defects  to  he  accepted. 


The  Air  Fore?  has  adopted  a  policy  of  assisting 
contractors  in  developing  their  own  quality  control 
programs.  Accordingly,  activities  who  issue  C  A  solicit a-iens 
enclose  a  copy  of  the  quality  assurance  plan.  This  enclosure 
is  marked  as  follows:  [  Ref .  39] 

For  Information  Purposes  Only.  This  Quality  Assurance 
surveillance  Plan  is  net  part  or  the  Request  for 
Proposal  cr  Invitation  For  Bins  nor  wv.l  it  be  made  Dart 
of  any  resulting  contract. 

A  Contract  Administrator  plan  can  also  be  enclosed 
with  the  solicitation,  describing  Contract  Administrator 
duties  in  evaluating  the  performance  of  Quality  Assurance 
Evaluators  (who  inspect  the  contractor).  Contract 

Administrators  may  also  make  random  inspections  of  contract 
per  f crmance . 

Contractors  are  provided  appropriate  pages  from 
HIL-STD-105E  to  assist  them  in  establishing  their  own 
quality  control  systems.  Instructions  are  provided  which 
describe  how  extrapolated  deductions  will  be  made  when 
specified  AQL' s  are  exceeded.  The  contractor  is  never  given 
schedules  cf  inspection  which  have  oeen  developed  from  the 
random  number  tables. 

The  Air  Force  approach  appears  to  be  one  of  reason¬ 
ableness.  Conversations  with  personnel  at  two  Air  Force 
bases  revealed  that  performance  rendered  under  CA  contracts 
with  statistical  surveillance  methods  is  very  satisfactory. 
Host  problems  occur  in  the  initial  transition  periods  of 
contracts  (the  first  one  or  two  months  of  performance)  . 
AQL's  for  mest  contracts  at  both  installations  have  rarely 
been  exceeded.  Government  personnel  seemed  to  be  pleased 
with  the  results  of  random  sampling  inspection  methods. 
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Seme  deviations  from  MIL-STD-1 05D  are  detected 


AFR  400-28.  No  provision  is  made  for  -lightened  inspections, 
as  single  samp lira  tables  in  the  regulation  address  only 
normal  (Level  II  :  average  quality)  and  reduced  (Level  I  : 
good  quality)  inspection. 

Switching  from  normal  to  reduced  inspection  under 
AFR  400-28  is  allowed  whan  four  consecutive  lets  have  been 
accepted,  and  the  number  of  defects  is  less  than  one  half  of 
the  specified  acceptance  number  for  normal  inspection.  Ihe 
division  manager  and  contract  administrator  must  also  agree 
tc  the  switch.  MIL-SID-105D  allows  this  only  after  ten 
consecutive  lots  have  been  accepted.  A  return  tc  normal 
inspection  will  be  implemented  if  the  acceptance  number  £c 
reduced  inspection  is  exceeded.  (J.nder  MI  L-STD- 1 0SD ,  a 
switch  to  tightened  inspection  is  necessary  when  two  of  five 
consecutive  lots  are  rejected.  A  switch  from  tightened  Dack 
to  ncrmal  inspection  is  allowed  when  five  consecutive  lots 
are  accepted.  AFR  400-28  does  not  incorporate  a  similar 
provision . 

It  might  be  reasoned  that  these  Air  Force  deviations 
are  meant  tc  offer  positive  motivation  to  contractors  in 
performing  satisfactorily.  These  modified  procedures  are 
being  utilized  at  most  Air  Force  activities  undergoing  A-7 b 
CA  contract  evolutions. 

E.  NAVFAC  USB  OF  CONFIDENCE  LEVEL  ESTIMATION  METHODS 

The  Naval  Facilities  Engineering  Command  (NAVFAC)  has 
been  heavily  involved  in  the  management  of  a  growing  number 
of  CA  contracts  for  facility  related  services  a-  Navy  activ¬ 
ities.  It  is  or.e  of  the  first  systems  commands  under  the 
Chief  of  Naval  Material  (CNM)  to  address  diverse  issues  in 
CA  conversions  and  begin  tae  formulation  of  policy  to  isai. 
with  them. 
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NAVFAC  published  the  MO-327  to  offer  guidance  oo  Navy 
activities  in  preparing  performance-oriented  statements  of 
work  and  quality  assurance  plans.  This  guidance  is  similar 
to  that  provided  in  Office  cf  Federal  Procurement  Policy 
Pamphlet  Ho.  4,  and  allows  inspection  methods  ether  than 
random  sampling.  NAVFAC  has  issued  20  standardized  state¬ 
ments  of  work  with  matching  quality  assurance  plans  which 
can  be  tailored  by  Navy  activities  to  incorporate  specific 
local  requirements.  Portions  of  MO-327  are  illustrated  in 
Appendix  C. 

NAVFAC  has  net  adopted  the  use  of  MIL-STD-105D  as  the 
basis  for  its  statistical  sampling  techniques,  and  does  not 
yet  allow  Navy  activities  tc  maxe  extrapolated  deductions 
based  on  its  usage.  Sampling  techniques  may  be  used  at 
those  Navy  activities  which  demonstrate  the  ability  to 
establish  sophisticated  quality  assurance  programs.  New  PWC 
transportation  CA  solicitations  being  issued  in  this  fiscal 
year  will  include  modified  sampling  techniques  and  extrapo¬ 
lated  deductions  provisions  based  on  their  usage. 

NAVFAC  philosophy  is  that  CA  contract  surveillance 
should  be  based  on  the  statistical  estimation  of  defective 
items  in  samples,  rather  than  on  the  accept/reject 
hypothesis  testing  methodology  of  MIL-STD- 1050.  Estimation 
is  intended  to  inject  a  higher  level  of  precision  (confi¬ 
dence)  in  the  surveillance  process  than  that  offered  by 
MIL  -  STD-  1 05  C . 

This  methodology  requires  a  designation  of  desired 
confidence  levels,  a  relative  accuracy  cf  estimation,  popu¬ 
lation  sizes,  and  a  threshold  of  conformance  (equivalent  to 
the  ACL  concept).  Combining  these  elements  results  in  a 
determination  of  sample  sizes  and  corresponding  lower  confi¬ 
dence  limit  rejection  numbers.  For  example,  a  work  function 
is  performed  2000  times  in  a  month.  A  confidence  level  cf  95 
percent  is  desired  in  estimating  the  number  of  population 
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defects,  with  a  relative  accuracy  equal  to  50  percent  of  the 
designated  AQL  cf  10  percent.  (Relative  accuracy  then  equals 
5  percent).  A  sample  size  of  93  items  is  required,  and 
rejection  of  the  month’s  performance  can  occur  if  mere  than 
16  defective  items  are  found.  Sixteen  observed  defects  will 
result  in  proper  let  rejection  since  the  estimated  lower 
confidence  limit  percentage  for  nonconforming  items  is 
always  greater  than  the  specified  AQL  of  10  percent. 

NAVFAC  has  worked  with  the  Office  of  Naval  Research 
(ONR)  in  developing  a  sat  of  new  confidence  level  estimation 
tables  which  can  be  used  to  design  CA  contract  quality  assu¬ 
rance  systems.  These  tables  will  reflect  three  different 
levels  of  inspection  intensity  (corresponding  to  various 
confidence  levels),  known  as  tightened  (99  percent  confi¬ 
dence),  normal  (95  percent),  and  reduced  (90  percent) 
sampling.  It  appears  that  these  new  tables  are  based  cn  the 
hyper  gee  me* ric  statistical  distribution. 

This  new  methodology  emanated  since  NAVFAC  doubts  that 
MIL- S ID-  1 0SD  suitably  estimates  fractions  of  nonconforming 
activities  with  reasonable  accuracy.  This  was  illustrated 
earlier  when  observed  defect  rates  at  the  HIL-SID-1 C5D 
reject  number  were  compared  to  the  AQL's.  NAVFAC  also 
desires  flexibility  in  choosing  AQL's  other  than  these 
provided  in  the  standard's  tables. 

A  primary  emphasis  in  confidence  level  estimation  is  to 
determine  the  actual  occurrence  of  defects  in  the  population 
based  on  sample  observations  whereas  hypothesis  testing  by 
attributes  only  determines  if  populations  are  acceptable  or 
unsatisfactory . 

NAVFAC  has  ether  reservations  concerning  the  use  of 
MIL-STD- 105 E.  It  does  not  believe  that  full  payments  should 
be  mads  when  the  observed  defect  rate  exceeds  the  AQL 
percentage,  even  if  the  reject  mummer  has  not  been  exceeded, 
and  guesticr.s  the  use  of  the  standard  where  lots  may  net  be 
homogenous  and  be  submitted  in  a  continuous  manner. 


Other  differences  in  Air  Force  and  NAVFAC  ccr. 


policy  were  observed  daring  the  study.  The  Air  Force 
encloses  copies  of  quality  assurance  plans  with  CA  contract 
solicitations;  NAVFAC  contracting  activities  have  nor  dene 
so.  Air  Force  contractors  are  notified  of  desired  AQL '  s  and 
the  type  cf  inspection  method  used  in  verifying  these. 
NAVFAC  will  provide  this  information  tc  prospective  bidders 
in  its  future  CA  solicitations.  The  Air  Force  administers  a 
management  control  program  in  which  cognizant  Contract 
Administrators  conduct  random  checks  of  both  ins  sect  or  and 


T  ABLE  I 

Samples  Sizes  For  HIL-STD-1Q5D  and  Confidence  Intervals 


Lot  Size 

Sample  sizes 

MIL-  STD- 1  0  5D  Confidence 

Inte 

50 

8 

37 

100 

20 

58 

5C0 

50 

108 

1000 

80 

121 

5000 

125 

135 

10000 

125 

136 

These  samples  sizes  are  for  normal  inspection  at 
953  confidence,  at  an  AQL  of  10*. 


contractor  performance.  NAVFAC  has  considered  such  a 
program  but  has  not  formerly  instituted  one. 

The  most  noticeable  difference  in  inspection  philoso¬ 
phies  is  visible  if  sample  sizes  of  each  are  compared  over 
increasing  let  sizes,  as  shewn'  in  Table  I.  Sample  sizes 
under  MIL-STD-105D  are  less  than  those  for  confidence  level 
estimation  for  smaller  populations.  wo  inferences  can  oe 


drawn  from  this  fact. 


If  cost  savings  are 


para,  me  u.t 
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importance  in  conducting  CA  contract  surveillance,  riven 
MIL-STD-105D  sampling  tables  should  be  utilized  because 
smaller  numbers  of  required  inspections  result  in  less  costs 
to  the  activity.  However,  if  precision  in  adjudging 
contract  performance  is  essential,  then  sampling  under 
confidence  level  estimation  should  be  utilized.  The  larger 
sample  size  provides  greater  reliability  in  assessing  the 
number  cf  defects  that  might  be  submitted.  If  costs  of 
inspection  are  net  great,  this  method  is  preferable. 

In  one  study  at  the  Charleston  Naval  Shipyard,  random 
sampling  procedures  were  used  successfully  for  daily,  repet¬ 
itive  services.  All  inspection  personnel  found  these  to  be 
superior  tc  former  heavy  reliance  on  planned  sampling 
methods.  Additionally,  use  of  such  techniques  resulted  in 
significant  lowering  cf  observed  iefect  rates  in  contractor 
performance. 

The  authors  were  told  that  an  informal  DO 0  wcrJcir.g  panel 
of  Air  Force,  Navy,  and  OFPP  personnel  will  begin  ar.  evalua¬ 
tion  cf  different  available  statistical  methods  when  the  new 
CNR  sampling  tables  are  complete.  Until  then,  the  only  offi¬ 
cial  statistical  method  available  for  use  is  MIL-STD-105D. 

P.  USING  STATISTICAL  SAMPLING  RESULTS  FOR  EXTRAPOLATED 

PAYMENT  DEDUCTIONS 

As  well  as  promoting  more  optimum  use  of  costly  inspec¬ 
tion  resources,  SQC  techniques  provide  another  important 
benefit.  Their  results  can  be  utilized  in  extrapolating,  or 
applying  the  percentage  of  defectives  found  in  samples  tc 
the  larger  let  populations. 

The  Air  Force  first  mandated  this  procedure  in  AF3 
400-26.  fchsn  sample  errors  occur  at  a  rate  which  is  or  eater 
than  the  desired  AQL  and  its  corresponding  reject  number,  a 
percentage  -found  unacceptable  results.  It  is  subtracted  from 


100  percent  to  determine  the  lot  percentage  acceptable  for 
payment,  which  is  applied  tc  the  entire  lot.  Deductions  are 
not  taken  when  errors  in  a  sample  are  less  than  the  reject 
number. 

NAVFAC  policy  has  not  allowed  extrapolated  deductions 
since  it  has  no  official  statistical  method  on  which  to  case 
these  deductions;  however,  progress  in  developing  sucn 
methods  has  led  NAVFAC  to  envision  alternative  applications 
cf  extrapolated  deductions  to  be  utilized  when  a  uniform  DOD 
statistical  ampiing  policy  is  adopted.  Appendix  Z  illus¬ 
trates  various  Air  Force  and  NA 7 F AC  methods  cf  making 
payment  deductions  fcr  nonconforming  service.  The  N  AV FAC 
methods  have  not  yet  been  promulgated  as  new  CA  contract 
policy.  It  should  ts  noted  that  NAVFAC  will  deduct  fcr  all 
items  observed  to  be  in  nonconformance.  When  contractors 
satisfactorily  r eper form  these  within  allowable  time  frames, 
however,  credit  will  be  given  for  reperformed  items, 
liquidated  damages  are  assessed  as  a  percentage  of  all 
nonccnfcr ming  items  cf  work. 

Promulgation  of  a  uniform  DOD  statistical  sampling 
policy  will  play  an  important  role  in  allowing  tae  use  of 
extrapolated  deductions  based  upon  this  technique.  If 
sampling  procedures  are  perceived  to  oe  unreliable,  litiga¬ 
tion  proceedings  may  eventually  prohibit  the  use  cf  this 
deduction  methodology. 

G.  AS  OVERVIEW  OF  SQC  TECH  HIQOES  FOH  A-76  CONTRACTS 

It  should  be  apparent  that  the  use  cf  ran  1cm  sampling 
procedures  in  the  new  A-76  Commercial  Activities  program  for 
COD  and  other  federal  agencies  offers  significant  cost 
savings  in  contract  administration.  Inspection  costs  are 
reduced,  ar.d  incorporation  cf  SQC  techniques  may  lower  the 
costs  that  cent ractors  incur  in  performing  such  services. 
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These  "Techniques  offer  fairness  and  objectivity  that 
often  were  not  present  under  former  planned  samplir.a  and 
customer  complaint  programs.  If  SQC  techniques  are  adminis¬ 
tered  properly,  they  will  render  a  much  more  accurate 
presentation  of  true  contractor  performance  than  was 
possible  under  former  inspection  methods. 

Such  techniques  will  be  invaluable  if  large,  multifunc¬ 
tion  contracts  at  federal  activities  become  a  reality  (such 
as  the  octal  base  operating  services  contract  at  the  Naval 
Submarine  Base  Bangor,  Washington).  These  will  optimize  the 
use  cf  minimal  government  contract  administration  resources. 


V.  PLANNING  AND  EODGET ING  FOR  CONTRACT  INSPECTION 

ORGANIZATIONS 

A.  INTRODUCTION 

With  the  implementation  of  ONE  Circular  A-76,  DOC  activ¬ 
ities  that  study  CA  functions  must  prepare  for  the  possi¬ 
bility  of  a  contract  award  by  identifying  and  planning  for 
certain  support  requirements  that  would  be  r.ece ssitated. 
Such  plans  include  a  quantification  of  inspection  resources 
so  that  the  activity's  budget  can  accurately  reflect  fund 
requirements  for  either  the  creation  of  a  service  contract 
organization  or  for  the  augmentation  of  an  existing  one. 

The  Commander,  Naval  Facilities  Engineering  Command 
(NAVFAC)  and  the  Commander,  Naval  Supply  Systems  Command 
(NA  VS  UP)  have  traditionally  held  contract  authority  for 
construction  and  procurement  of  supplies.  Both  have  dele¬ 
gated  contract  authority  to  regional  procurement  commands 
such  as  NAVFAC's  Engineering  Field  Divisions  (EFD's), 
NAVSUE's  Navy  Regional  Contracting  Centers  (NRCC's),  and 
Naval  Supply  Centers  (NSC's).  These  commands  have  further 
delegated  contract  authority  to  field  activities.  Along  with 
this  authority  ,  field  commands  have  been  assigned  the 
responsibility  for  quality  assurance  and  surveillance  of 
construction  and  delivered  supplies  .  In  the  Naval 
Facilities  Engineering  Command,  these  are  known  as  Resident 
Officers  In  Charge  of  Construction  (ROICC)  .  These  field 
activities  have  been  staffed  with  necessary  quality  assu¬ 
rance  an  d  contract  specialist  personnel  to  administer 
construction  and  supply  contracts.  Additionally,  the 
responsibility  for  the  preparation  of  contract  solicitation 
packages  (for  example,  the  design  of  constru ctior.  projects) 
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has  usually  been  assigned  to  the  regional  contracting  office 
cr  to  the  requiring  activity. 

This  pattern  of  contract  authority  and  surveillance 
responsibility  does  net  apply,  however,  for  CA  service 
contracts.  Neither  NAVFAC  nor  NAVSUP  have  been  provided  with 
sufficient  staffing  resources  to  provide  on-site  surveil¬ 
lance  and  administration  of  contracts  resulting  from  A-76 
cost  studies,  although  both  systems  commands  retain  contract 
authority  fer  CA  contracts.  Navy  activities  that  receive 
services  under  these  contracts  prepare  the  statements  of 
work,  quality  assurance  plans,  and  provide  an  cn-site 
surveillance  organization  to  ensure  proper  contractor 
performance . 

A  Chief  of  Naval  Operations  message  that  was  promulgated 
on  22  November  1982  delineated  different  roles  for  both 
contracting  agent  commands  and  customer  activities  in  their 
implementation  of  the  CA  program.  The  following  responsi¬ 
bilities  are  identified;  [  Bef.  37] 

a)  Contracting  Office 

i)  Processing  of  contract  documents  which  requires 
exercise  cf  contract  authority, 

ii)  Negotiation  of  ail  contract  changes, 

iii)  Direction  of  remedial  contractor  action, 

iv)  The  processing  cf  contractor  payment  requests, 

v)  Delegation  of  authority  to  the  customer  activity 
for  any  cay-tc-day  surveillance  of  the  contrac¬ 
tor’s  work  performance. 


Maintenance  of 
process. 


integrity 


throuahe ut 


vii)  Provision  cf  all  technical  advice,  and 

viii)  Provision  of  assistance  in  the  development  of 
training  programs  for  ail  personnel  involved  in 
contract  precaraticn  and  administration. 
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b)  Customer  Activity 

i)  Provision  of  qualified  personnel  to  inspect  work, 
delivered  under  the  contract, 

ii)  Preparation  and  implementation  of  quality  assu¬ 
rance  plans, 

iii)  Submission  of  quality  assurance  summary  reports 
to  the  contracting  agent, 

iv)  Evaluation  of  contractor  requests  for  payment, 

v)  He co mm en cation  cf  payments  or  deductions  to  the 
contracting  agent, 

vi)  Submission  cf  cost  estimates  to  the  contracting 
officer  for  proposed  modifications, 

vii)  Provision  of  assistance  as  required  to  the 
contracting  agent  during  modification  negotia¬ 
tions,  a  r.  d 

viii)  Performance  of  other  service  contract  support 
duties  when  these  are  delegated  by  the 
contracting  office. 

navfac  has  alerted  all  Navy  field  activities  and  major 
claimants  cf  this  division  in  service  contract  responsibili¬ 
ties  in  both  an  instruction  ( NAVFACI NST  4330.45)  and  a  new 
maintenance  manual  (MC-327)  .  The  Air  Force  has  adopted  a 
similar  approach  which  is  specified  ir.  two  separate  Air 
Force  Regulations  (AFR  400  -28  and  AF 8  70-9).  They  require 
that  the  functional  divisions  of  each  Air  Force  base  prepare 
toth  ths  statement  cf  work  and  corresponding  quality  assu¬ 
rance  plan.  The  division  must  provide  all  necessary  quality 
assurance  personnel  and,  prior  to  the  contract  solicitation, 
certify  in  writing  to  the  base  Commanding  Officer  that  these 
personnel  have  been  designated,  trained,  and  dedicated 
solely  tc  perform  surveillance  functions.  [Hef.  38] 

At  two  Air  Force  installations  visited  during  the 
research  phase  of  this  thesis,  it  was  discovered  that  a 
consolidated  steering  committee  is  organized  upon  the  formal 
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announcement  of  a  C A  cost  study.  It  includes  representa¬ 
tives  from  all  departments  that  mignt  be  affected  by  the 
contracting  cut  of  a  function  and  tailors  standardized  Air 
Force  statements  of  work  that  have  been  prepared  for  each  CA 
function,  identifying  and  including  ail  special  local 
command  requirements.  The  steering  group  aisc  provides 
assistance  in  the  development  of  the  guality  assurance  plan. 

B.  THE  ESTIMATION  OF  REQUIRED  INSPECTION  EESOORCES 

At  the  cutset  cf  a  cost  study  announcement,  an  activity 
must  begin  to  define  and  plan  its  guality  assurance  require- 
ments.  [Hef.  39]  The  planning  and  budgeting  responsibility 
for  obtaining  inspection  resources  belongs  to  the  activity. 
Each  Navy  activity  should  immediately  begin  the  planning  for 
a  contract  administration  staff,  establish  ongoing  interface 
with  the  appropriate  major  claimant  to  document  these 
requirements,  estimate  future  budget  amounts,  and  obtain 
ceiling  points  to  facilitate  the  creation  of  this  vital 
organ ' nation . 

Various  procedures  have  been  used  for  determinina  the 
required  quantity  cf  service  contract  administration 
resources.  In  the  supplement  of  the  1979  revision  of  A- 76, 
it  was  stated  that  ccsts  of  administration  resources  were  to 
be  estimated  to  be  four  percent  of  the  projected  award  cost 
cf  a  service  contract.  This  factor  is  useful  only  as  an 
approximate  estimate  of  inspection  requirements.  Actual 
inspection  costs  migat  range  from  ten  percent  for  small 
service  contracts  to  only  two  percent  for  large  50S 
contracts.  Use  of  this  simple  estimating  factor  does  net 
account  fer  differing  degrees  of  complexity  among  contracts; 
however,  it  may  be  satisfactory  as  an  initial  estimating 
tool .  [  Hef .  5 ] 
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Another  simple  estimating  model  for  determining  inspec¬ 
tion  requirements  has  been  recommended  by  the  the  Scuta  err. 
Engineering  Field  Division  (SOUTHDIV)  of  the  Naval 
Facilities  Engineering  Command.  It  recommends  that  activi¬ 
ties  can  plan  to  use  375  hours  of  inspection  for  every 
$300,000  dollars  of  refuse  collection  service  contracts, 
while  all  ether  service  contract  types  require  at  least 
1,325  hours  of  inspection  for  every  $300,000  of  contract 


TABLE  II 

New  A-76  Contract  Administration  Factors 


In- House  Staff  fes i 
Eeina  Studied 


irons  Contract  Administration 
Staff  Requirements 

staff 


Be  low 

10 

Use  existin 

10  - 

2G 

1 

21  - 

42 

2 

43  - 

65 

3 

66  - 

9  1 

4 

92  - 

1  19 

5 

120  - 

150 

6 

15  1  - 

i  e4 

7 

185  - 

2  22 

8 

22 J  - 

265 

Q 

266  - 

3  12 

1 6 

3  13- 

367 

1 1 

360  - 

429 

12 

430  - 

5  C  0 

13 

501  - 

563 

14 

584  - 

6  82 

15 

683  - 

80  0 

1  <k 

Above 

800 

2%  of  In- House  St 

costs.  This  model,  lake  the  previous  one,  as  primarily 
useful  for  making  rudimentary  projections  of  service 
contract  inspection  requirements. 

The  1933  proposed  revision  of  Circular  A-76  provides 
guidance  shewn  in  Table  II  that  relates  the  number  of 
personnel  required  fer  contract  administration  to  the  number 
of  personnel  positions  being  studied  for  conversion. 
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shows  hew  tc  estimate  the  required  contract  office 


staffing. 

This  particular  methodology,  as  the  two  previous  ones,  may 
not  account  for  differing  complexities  which  exist  in 
service  contracts. 


Z  =  69.74  ♦  (0.183)  (A)  +  (7.29)  (E)  ! 

Where  A  *  Total  Service  Contract  Value  (3,000)  I 

and  E  =  Total  t  of  Service  Contracts  I 

QAS's  required  =  C  livided  by  144  I 


Figure  5.1  Inspector  Hours  Regression. 

Another  method  for  estimating  the  number  of  required 
inspectors  is  found  in  the  Student  Guide  for  Maintenance 
Service  Contracts  published  by  the  Civil  Engineer  Corps 
Officers  School.  This  mathematical  metnod,  depicted  in 
Figure  5.1,  is  an  estimating  model  that  relates  the  number 
cf  required  inspectors  to  the  total  monetary  value  and 
number  of  the  activity's  service  contracts.  [Ref.  40] 

The  Atlantic  Engineering  Field  Division  (LANTDIV)  cf  the 
Naval  Facilties  Engineering  Command  has  proposed  an  alterna¬ 
tive  which  may  he  the  most  viable  one  recommended  thus  far 
and  is  provided  as  Appendix  F.  LANTDIV  Instruction  11014. 4D 
recommends  the  use  of  a  standardized  worksheet  to  accurately 
estimate  required  inspection  resources.  It  requires  the 
specific  quantities  cf  various  work  elements  to  be  performed 
under  the  contract  on  a  monthly  basis,  estimated  number  of 
inspection  hours  to  observe  these,  and  the  total  number  of 
hours  tc  inspect  individual  categories  as  well  as  the  total 
contract  performance  fer  each  month.  Tnis  estimation  will 
allow  a  determination  cf  the  required  number  of  inspectors, 
based  upon  a  total  estimate  of  inspection  manhours. 


Utilization  of  this  worksheet  should  result  ir.  tcre 
accurate  calculations  of  inspection  personnel  resources  that 
are  required  if  a  new  contract  is  awarded.  This  approach  ray 
also  lead  to  the  creation  of  sufficient  data  that  can  be 
used  in  the  development  of  engineered  performance  standards 
for  inspection  of  commercial  activities  service  contracts. 

If  any  of  the  above  estimating  models  prove  to  he  unsa¬ 
tisfactory,  then  the  safest  means  of  estimating  personnel 
requirements  may  be  to  predict  that  one  new  inspector  will 
be  necessary  for  each  new  function  to  oe  awarded.  This  may 
be  especially  true  if  the  inspection  forces  are  located 
under  a  functional  department,  as  is  often  the  case  for  Air 
Force  service  contracts. 

Inaccurate  estimates  of  required  staffing  may  create 
serious  problems  for  activities  that  continue  to  contract 
cut  under  A-76  guidelines.  Navy  activities  must  carefully 
design  all  of  their  quality  assurance  plans  ana  calculate 
the  required  number  cf  inspectors  and  staff  personnel  to 
execute  them.  Although  current  policy  dictates  that  defini- 
tized  quality  assurance  plans  and  organized  quality  assu¬ 
rance  staffs  will  be  established  before  contract 
solicitations,  research  for  this  study  indicated  that  formal 
gua-ity  assurance  plan  designs  often  lag  contract  award. 
Activities  that  were  visited  frequently  used  existing 
personnel  tc  provide  surveillance  on  the  new  CA  contracts. 

Action  pursued  in  accordance  with  these  considerations 
will  be  essential  to  safeguard  the  activity's  interest  in  a 
growing  trend  of  contracting  out  for  the  Navy's  commercial 
services.  If  this  evolution  is  not  planned  properly  ,  inef¬ 
fective  cr  erratic  government  surveillance  may  result.  Such 
a  condition  can  be  a  significant  factor  in  these  contract 
litigations  cr  disputes  which  arise  when  the  propriety  of 
government  inspection  is  a  major  issue  . 
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C.  QUALITY  ASSURANCE  TRAINING 


Up  -o  this  point,  it  has  been  emphasized  that  the  deter¬ 
mination  and  staffing  of  a  sufficient  number  cf  contract 
inspection  and  administration  personnel  is  vital  to  a 
successful  implementation  cf  A-76  objectives.  An  equally 
important  consideration  is  appropriate  training  cf  these 
perscnnel  who  will  be  involved  in  contract  surveillance. 
Even  if  no  additional  inspection  personnel  can  be  cDtained 
througn  budgetary  procedures,  the  activity  will  still  be 
required  tc  provide  a  cadre  cf  inspectors  from  existing 
personnel  assets  whe  will  need  to  oe  knowledgeable  in 
surveillance  techniques.  [Ref.  41] 

Curing  this  research,  it  was  discovered  that  twe  Air 
Force  activities  designate  and  train  Quality  Assurance 
Evaluatcr  (QA3)  candidates  during  the  statement  cf  work 
preparation  process  in  accordance  with  Air  Force  Regulation 
70-9.  Current  Navy  policy  has  left  tne  choice  of  inspectors 
and  training  responsibility  to  the  discretion  of  each  activ¬ 
ity's  Commanding  Officer.  Navy  guidance  in  this  area  (that 
of  NAVFAC  and  NAVSU?)  has  been  only  advisory  in  nature.  The 
cniy  written  mandatory  requirements  levied  upon  Navy  activi¬ 
ties  is  that  they  prepare  and  sunmit  a  quality  assurance 
plan  to  contracting  agents  for  approval  prior  to  solicita¬ 
tion;  they  must  also  certify  in  writing  that  a  quality  assu¬ 
rance  workforce  will  be  established  or  augmented  pricr  to 
the  contract  award.  Ihese  requirements  have  been  addressed 
in  recent  NAVFAC  Engineering  Field  Division  guidance  that 
pertains  to  CA  service  contracting. 

Newly  designated  Quality  Assurance  Evaluators  cannot  be 
expected  tc  immediately  and  professionally  execute  their 
surveillance  responsibilities  until  they  have  been  qualified 
by  means  cf  a  formal  training  process.  This  may  require 
attendance  at  a  special  school  designed  to  teach  various  CA 
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quality  assurance  r  esponsib  llities.  It  may  also  involve  ar. 
ongoing,  less  formalized  training  process  that  is  carried 
out  by  skilled  quality  assurance  specialists  who  reside  at 
cr  visit  the  activity. 

Service  contract  training  courses  have  been  designed  and 
are  beinc  taught  to  the  growing  number  of  federal  personnel 
who  perform  surveillance  functions.  For  the  Air  Force, 
training  is  conducted  by  a  quality  assurance  program  coordi¬ 
nator.  The  QA  Program  Coordinator  serves  in  a  general  advi¬ 
sory  capacity  and  monitors  the  performance  of  functional 
department  CAE's  after  they  complete  initial  training  ana 
begin  performing  surveillance  duties.  2A2  candidates  receive 
basic  training  in  contract  law  ar.d  administration,  quality 
assurance  duties  and  responsibilities,  and  an  overview  of 
the  quality  assurance  plan  .  The  training  responsibility  for 
Quality  Assurance  Evaluators  is  placed  at  the  activity  level 
under  Air  Force  procedures.  Quality  Assurance  Evaluators  are 
identified  and  trained  before  the  contract  solicitation 
process  begins  . 

Fcr  the  Navy,  Quality  Assurance  Evaluator  manning  is 
handled  by  the  activity  and  the  training  is  carried  cut  by 
the  nearest  Engineering  Field  Division.  The  Facilities 
Division  (cede  10)  of  each  NAVFAC  EFD  provides  this  training 
to  naval  activities  in  its  geographic  area.  Training  of  Navy 
QAE's  is  a  contracting  agent  responsibility.  ZFD's  also 
provide  technical  support  to  each  activity  during  the  state¬ 
ment  cf  werk  and  quality  assurance  development  processes. 

The  Naval  Facilities  Engineering  Command  has  developed 
its  own  training  manual  which  describes  the  specific  roles 
assigned  to  quality  assurance  evaluators.  This  manual  is 
known  as  MC-326.2,  "Quality  Assurance  Evaluators  Training 
Manual"  ard  is  utilized  oy  each  Engineering  Field  Division 
in  its  quality  assurance  training  presentations.  Quality 
assurance  evaluators  perform  the  following  duties  which  are 
lasted  in  MO-326  .2  :  [Ref.  42] 
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a)  Review  plans  and  service  contract  specifications 
before  contract  solicitation; 

b)  Assist  in  pre-award  surveys; 

c)  Attend  the  pre-bid  and  post-award  conferences; 

d)  Coordinate  transfer  of  government  furnished  space, 
utilities,  equipment  and  material  to  the  contractor; 

e)  Prepare  quality  assurance  plans; 

f)  Prepare  surveillance  schedules,  perform  surveillance, 
and  submit  reports  of  findings; 

g)  Review  all  contractor  schedules  and  advise  Service 
Contract  Manager  of  acceptability ; 

h)  Assist  in  the  preparation,  or  directly  prepare, 
government  estimates  for  change  orders; 

i)  Recommend  deductions  for  unsatis factory  work  to  the 
Service  Contract  Manager; 

j)  Mcnitor  the  contractor's  safety  practices  and  report 
results; 

k)  Conduct  labor  standards  interviews  as  necessary;  and 

l)  Conduct  surveillance  on  the  contractor's  accomplish¬ 
ment  of  required  corrective  changes. 

It  should  be  emphasized  that  this  list  is  not  exhaus¬ 
tive.  In  spite  of  an  implicit  policy  that  quality  assurance 
evaluators  should  be  dedicated  solely  to  surveillance  func¬ 
tions,  they  may  work  for  functional  managers  and  perform 
several  ether  duties  ir.  addition  to  tneir  surveillance 
responsibilities . 

When  considering  QA2  responsibilities,  it  is  evident 
that  a  well  organized  training  course  addressing  several 
elements  cf  contract  administration  is  essential. 
Additionally,  the  course  must  be  presented  in  a  clear  and 
concise  manner  using  terminology  that  is  easily  understan¬ 
dable  by  the  layman  .  Training  in  basic  contract  law, 
administration  principles,  cost  and  price  analysis,  organi¬ 
zation  structure,  and  contract  specifications  should  be 
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covered  .  An  equally  important  topic  which  must  be  included 
in  any  quality  assurance  courses  for  service  contracts  is 
statistical  quality  assurance.  As  discussed  earlier,  the 
use  cf  statistical  sampling  techniques  has  teen  advocated 
for  the  inspection  of  commercial  activities  service 
contracts.  It  offers  significant  advantages  over  the  use  of 
cr.e  hundred  percent  inspection  and  reliance  upon  customer 
complaints. 

Specific  training  in  the  use  of  Military  Standard  105D 
and  random  number  tables  is  required.  Inspectors  should  knew 
hew  tc  form  homogenous  lot  and  sample  sizes,  establish  real¬ 
istic  acceptable  quality  levels  (AQL's) ,  create  monthly 
randem  sampling  schedules,  and  conduct  inspections.  Finally, 
inspectors  will  require  instruction  in  properly  disposing  of 
unacceptable  variances  from  required  performance. 
Statistical  quality  assurance  may  initially  appear 
confusing,  tut  a  well  designed  training  program  will  reveal 
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Study  observations  revealed  than  several  Navy  activities 
have  attempted  to  send  Quality  Assurance  Evaluators  tc 
appropriate  training  courses  but  insufficient  funds  have 
prevented  attainment  of  this  goal.  It  was  observed  that 
on-site  training  programs  were  generally  non-exister.t,  and 
should  be  initiated  to  enhance  the  skills  and  capabilities 
of  Navy  Quality  Assurance  Evaluators. 

D.  INTEGRATING  RESOURCES  TO  BUILD  A  QA  PROGRAM 

Thus  far,  three  basic  building  blocks  for  an  effective 
guality  assurance  organization  have  been  discussed.  These 
are  the  obtaining  of  sufficient  personnel,  training  of 
QAE’s,  and  funding  of  inspection  resources.  If  any  of  *hese 
elements  are  inadequate,  the  quality  assurance  program  to 
support  A-76  contract  conversions  will  be  jeopardized. 
Perhaps  the  underlying  intangible  element  that  ties  all  of 
these  together  is  commitment.  The  activity  Commanding 
Officer,  the  base  personnel,  the  major  claimants,  and  the 
various  contracting  agencies  will  need  to  form  a  coalition 
to  accurately  plan  and  provide  for  the  proper  integration  of 
these  building  blocks.  In  the  case  of  the  Navy,  cooperative 
attempts  are  being  pursued  to  bring  about  successful 
contracting  out  . 

The  authors  found  wide  variations  in  these  efforts. 
Formal  design  and  implementation  of  quality  assurance  plans 
has  lagged  the  awards  cf  many  CA  contracts  and  many  contract 
inspection  systems  at  Navy  activities  are  not  yet  formal¬ 
ized.  Random  sampling  inspection  is  used  at  very  few  Navy 
activities  that  administer  service  contracts.  seme  reasons 
for  its  infrequent  usage  are  that  it  seems  to  be  too  compli¬ 
cated  and  that  extrapolated  deductions  are  not  yet  allowed 
for  Navy  contracts  utilizing  random  sampling.  These  Navy 
activities  visited  believed  that  the  formation  cf  a  field 
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contracting  system  fcr  service  contracts  that  approximates 
the  organization  for  construction  contracts  will  make  the 
administration  of  such  contracts  more  viable  and  effective. 

The  size  of  the  inspection  organization  and  the  sophist¬ 
ication  of  the  predesigned  quality  assurance  plan  determine 
the  adequacy  of  the  overall  contract  administration  system 
for  CA  contracts.  Adequate  staffing  and  a  scunc  inspection 
plan  will  lead  tc  successful  surveillance,  winning  the  coop¬ 
eration  and  support  of  both  the  contractor  ar.a  the  base 
communities  that  receive  services.  Tans  plan  will  optimize 
the  use  of  inspection  resources.  Effective  scheduling  of 
inspectors  and  audits  of  contract  administration  can  be 
achieved.  An  alternative,  less  optimal  approach  is  tc  allow 
quality  assurance  planning  to  be  limited  by  inspection 
resources  that  are  available  to  the  activity.  It  is  likely 
that  this  option  will  be  the  one  most  often  chosen  if  activ¬ 
ities  dc  net  perform  the  necessary  pre-contract  planning  and 
budgeting  fer  the  formulation  of  CA  contract  organizations. 
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VI.  ORGANIZATIONS  FOR  A- 76  SERVICE  CONTRACT  MANAGEMENT 

A.  MANAGEMENT  OF  SERVICE  CONTRACTS  AT  NAVFAC  HEADQUARTERS 

Preceding  discussions  have  sen  the  tone  for  the  predomi¬ 
nant  issue  of  this  chapter,  the  makeup  ana  location  of 
service  contract  inspection  and  administration  organiza¬ 
tions.  It  is  first  appropriate  to  examine  the  organizational 
structure  for  management  of  service  contracts  at  NAVFAC 
headguarters. 

NAVFAC  has  located  management  of  service  contract  policy 
for  facility  support  contracts  with  the  Assistant  Commander 
for  Maintenance  and  Transportation  (code  10)  .  This  is  also 
true  for  the  management  of  service  contracts  at  the 
Engineering  Field  Division  level,  where  a  similar 
Maintenance  Division  (code  10)  oversees  the  evolution  of 
Navy  CA  service  contracting.  Authority  for  all  contracts 
(Commercial  Activities  or  construction)  is  vested  in  the 
Acquisition  Department  (code  09A)  at  both  headquarters  and 
EFD  levels.  A  typical  Engineering  Field  Division  performs 
the  following  management  and  contract  duties  for  A-76 
procurement:  £Ref.  *13] 

a)  Distributes  contracting  directives  and  provides 
guidance  to  field  contract  offices; 

b)  assists  activity  contract  offices  during  contract 
d  i  s  p  u  t  as ; 

cl  oversees  the  operation  of  activity  contract  offices  tc 
ensure  the  integrity  of  CA  service  contract 

processes  ; 

a)  reviews  activity  requests  fcr  sole  source  procurements 
and  refers  these  to  NAVFAC  when  appropriate; 
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e)  acts  as  the  central  point  of  contact  in  the  region  for 
all  CA  service  contract  matters; 

f)  maintains  a  technical  library  tc  support  compliance 
with  the  A-76  program,  and  provides  technical  informa¬ 
tion  tc  customer  activities; 

g)  assists  activities  in  statement  of  work  and  quality 
assurance  plan  development; 

h)  conducts  QAE  training  programs;  and 

i)  reviews  each  activity's  statements  of  work  and  corres¬ 
ponding  quality  assurance  plans  before  contract  soli¬ 
citation. 

Ey  comparison,  overall  management  of  the  Air  Force  CA 
service  contract  pregram  is  conducted  at  the  headquarters 
level  by  the  Air  Force  Service  Contract  Advisory  Group 
(AFSCAG)  .  Intermediate  systems  commands  such  as  the  Military 
Airlift  Command  (MAC)  or  Air  Force  Logistics  Command  ( AFLC) 
may  publish  amplifying  guidance  to  supplement  two  primary 
Air  Force  regulatory  policies  for  service  contracts,  AFR 

400-28  and  AFR  70-9,  as  shown  in  Appendix  H .  (Ref.  44], 

Inis  comprehensive  management  is  also  visible  at  the 

activity  level  when  functional  divisions  develop  statements 
of  work  suitable  for  the  activity's  requirements. 
Intermediate  systems  commands  grant  final  approval  to 
activity  statements  cf  worn  before  issuance  of  contract 
solicitations. 

In  late  1982,  a  few  cf  NAVFAC's  Engineering  Field 
Divisions  tegan  to  consider  the  consolidation  of  ail 

contractual,  planning,  training  and  technical  ipport  for  CA 
service  contracts  in  one  Facility  Support  Contract  3ranch. 
This  head  cf  ohis  branch  may  be  delegated  contractual 
authority  that  is  separate  and  distinct  from  that  authority 
held  by  the  department  head  for  facilities  construction  and 
may  be  able  to  further  delegate  CA  contract  authority  to 
responsible  cognizant  individuals  at  one  field  level  (such 
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as  the  field  activity’s  Public  Works  Officer).  This  concept 
is  still  under  study,  offering  striking  possibilif ies  for 
the  future  administration  of  NAVFAC's  service  contracts. 
Eerhaps  NAV FAC  will  consider  the  creation  of  a  separate 
service  contract  organization  that  is  an  exact  likeness  of 
the  one  for  construction  since  the  staff  and  expertise  to 
manage  such  a  program  is  largely  in  existence  at  NAVFAC 
headquarters,  Engineering  Field  Divisions,  and  the  large 
Public  Works  Centers.  Such  a  new  organization  would  require 
the  establishment  of  field  activity  service  contract  offices 
to  prcmoxe  uniformity  in  administration  of  A-76  contracts 
and  establish  management  control  of  the  service  contract 
process,  and  in  turn  promote  NAVFAC's  credibility  as  a 
service  contract  management  agent. 

B.  THE  ACTIVITY  ORGANIZATION  STRUCTURE 

1 .  Introduction 

In  the  current  A-76  setting,  most  commercial 
functions  are  related  to  facilities  or  supply  management. 
Accordingly,  the  officials  who  are  responsible  for  these 
areas  at  most  naval  activities  are  either  the  Public  Works 
Officer  or  the  Supply  Officer.  Such  functions  are  vital  to 
successful  accomplishment  of  the  activity  mission,  and 
failure  fc  provide  them  in  a  timely,  cost  effective  manner 
could  seriously  jeopardize  the  operating  posture  of  the 
activity.  Most  functions  are  currently  performed  by  federal 
civilians  cr  military  personnel,  but  growing  trends  in 
contracting  cut  under  the  A-76  policy  signal  a  change. 

2 •  I y p i ca  1  Public  Works  Department  Funct ions 

A  sample  of  the  different  roles  performed  at  a 
typical  Navy  Public  Works  Department  is  as  follows: 
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Ar.  important  sea?  in  the  A-76  service  contract 
process  is  the  choice  cf  a  suitable  organizational  structure 
to  manage  the  activity's  contract  administration  responsi¬ 
bilities.  NAVFAC  has  left  the  choice  of  the  size,  location, 
and  professional  skill  capabilities  for  this  new  organiza¬ 
tion  to  the  discretion  of  the  activity  Commanding  Officer. 
He  is  tasked  to  provide  surveillance  since  contracting 
agents  have  not  been  given  sufficient  funds  or  personnel 
resources  to  manage  CA  contracts.  Before  the  Ccmm ending 
Officer  builds  such  an  organization,  he  should  be  informed 
of  these  factors  which  should  influence  its  design. 

Traditional  werk  planning,  control  and  pest  perfor¬ 
mance  inspection  of  Public  Works  maintenance  and  service 
operations  have  been  performed  by  the  Maintenance  Control 
Division  (KCD)  of  the  Public  Works  Department  (possibly 
.excluding  utilities  and  transportation  operations).  MCD 
inspectors  prepare  jcb  orders  for  shops  personnel  and  esti¬ 
mate  time  and  material  requirements  based  upon  engineered 
performance  standards.  Inspectors  periodically  visit  jcb 
sites  tc  verify  proper  performance  of  repairs  and  mainte¬ 
nance.  They  also  compare  actual  job  expenditures  with  orig¬ 
inal  estimates.  As  CA  service  contracting  became  part  of 
Navy  facilities  management,  inspection  responsibilities  have 
been  assigned  tc  Maintenance  Control  Division  personnel. 
Initial  types  cf  contracts  inspected  include  custodial 
services,  grounds  maintenance,  refuse  collection,  and 
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housing  facilities  maintenance.  Sometimes  this  responsi¬ 
bility  is  shared  between  MCD  insoectors  an  i  case  tenants 
when  the  vclume  of  contracted  service-,  increases  signifi¬ 
cantly.  This  shared  responsibility  has  resulted  in  somewhat 
tencus  CA  contract  administration. 

Reliance  upen  base  customers  to  provide  required 
surveillance  is  not  always  a  suitable  practice.  There  is 
always  a  danger  that  customers  acting  m  a  contract  inspec¬ 
tor's  role  will  exercise  implied  or  apparent  authority  in 
dealing  with  the  contractor .  Constructive  changes  cr  unau¬ 
thorized  changes  in  scope  can  lead  to  counterproductive 
disputes  and  litigation;  these  dc  not  serve  the  best  inter¬ 
ests  cf  either  the  government  or  the  contractor.  where 
customer  surveillance  methods  are  used  in  a  way  that 
contract  performance  standards  are  ultimately  raised,  then 
relief  will  generally  be  provided  tc  the  contractor  under  a 
changes  cr  disputes  process.  A  surveillance  system  wnich 
depends  heavily  upon  customer  observation  of  a  contractor  is 
not  desirable  for  use  in  the  CA  service  contract  quality 
assurance  process.  (Ref.  42] 

Increasing  work  volumes  attributable  to  service 
contract  conversions  maxe  establishment  of  centralized 
service  contract  management  organizations  advantageous  to 
successful  CA  contracting.  Such  organizations  are  separate, 
distinct  station  divisions,  act  as  key  monitors  of  service 
contractor  activity,  and  will  probaoly  be  located  within 
Public  works  or  Supply  Departments.  The  CA  contract  inspec¬ 
tors  in  such  divisions  possess  a  wide  range  of  technical 
skills  sc  that  full  contract  surveillance  can  be  provided. 
This  arrangement  enhances  management  centre!  and  maintains 
integrity  in  the  contract  process. 

If  a  centralized  organization  (with  ail  QAI’s 
working  under  one  quality  assurance  manager)  is  net  attai¬ 
nable,  then  other  less  optimal  organization  arrangements 
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exist.  These  are  illustrated  and  explained  in  Figures  g. 1 
through  G.4  in  Appendix  G.  These  figures  show  than  a 
Facility  Support  Contract  Office  can  administer  CA  service 
contracts  of  a  facilities  nature,  being  located  as  a  sepa¬ 
rate  division  of  the  Public  Works  Department  (FWD)  or  as  a 
subdivision  of  the  PWD  Maintenance  Control  Division.  QAZ's 
may  work  in  the  proposed  central  organization,  or  may  work 
for  functional  division  managers.  [Ref.  45] 

4.  The  Nav^  Service  Contract  Manager 

A  key  player  in  NAVFAC's  service  contract  process  is 
the  Service  Contract  Manager  (SCM),  the  head  representative 
cf  the  activity's  quality  assurance  team.  His  role  is 
described  as  follows:  [Ref.  28] 


The  Service  Contract  Manager  is  that  oersor.  with  direct 
responsibility  for  day-to-day  managemement  of  the 
service  contract.  Prior  to  award  he  is  responsible  for 
assisting  in  the  preparation  of  the  statement  of  work, 
the  government  estimate,  and  the  surveillance  plan. 
Post  award  responsibilities  are  to  ensure  that  the 
contract  runs  smocthly  and  is  properly  managed,  that 
surveillance  is  conducted,  ana  documented,  that  contract 
working  files  are  maintained .  and  that  work  orders  are 
properly  coordinated  with  the  Of ficer-in-Charae.  if 
chanae  orders  are  required,  the  SCM  must  otr cess  them 
and  "  make  a  recommendation  *  to  the 

Resident-Cff icer- in-Charge  to  issue  a  change;  if  the 
contractor  is  having  problems,  the  SCM  must  recommend 
reguired  action  to  the  Resident-Off icer- in-Charge  in 
matters  involving  quality,  time,  money,  or  safety,  and 
must  coordinate  these  matters  with  tne  contractor,  the 
contract  specialist,  and  the  Resident-Of ficer-in-Char ae . 

The  quality  assurance  program  provides  tne  Service 
Contract  Manager  witn  information  on  the  contractor's 
performance.  The  SCM  has  technical  and  supervisory 
responsibility  for  this  program. 


his  individual  may  be  either  an  Engineering  Field 
Division  asset  or  may  te  on  the  field  activity's  rolls,  and 
possesses  an  appropriate  level  of  contract  authority.  The 
SCM  is  skilled  in  the  use  of  statistical  quality  assurance 
and  is  able  to  plan  and  manage  random  sampling  surveillance. 
He  probably  has  a  public  relations  rola.  Navy  SCM'S  may  be 
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either  military  officers  or  civilians.  For  the  Air  rcrce, 
this  player  is  known  as  the  Quality  Assurance  Evaluator 
Program  Coordinator;  the  major  difference  between  Air  rcrce 
and  Navy  roles  is  that  the  SCM  has  supervisory  responsi¬ 
bility  ever  QAZ's,  while  the  QAZ  Program  Coordinator  poss¬ 
esses  advisory  responsibility  only. 

5*  A  Choice  Of  Can  trallzed  or  Decentralized 

Organizations 

Ir.  creating  the  CA  service  contract  management  orga¬ 
nization,  several  alternatives  may  be  considered,  ranging 
from  decentralized  tc  centralized  formats.  The  centralized 
organization  has  already  been  discussed;  the  Service 
Contract  Manager  is  completely  in  charge  of  this  type  of 
structure.  Contract  specialists  and  quality  assurance 
personnel  report  to  the  SCM  who  usually  is  a  division  head 
reporting  tc  the  activity  Public  '.ierks  Officer  or  Supply 
Officer . 

At  the  ether  extreme,  the  SCM  is  a  staff  advisor 
with  partial  supervisory  responsibility  over  inspectors, 
with  all  quality  assurance  personnel  being  assigned  to  func¬ 
tional  department  managers.  The  SCM  may  find  it  difficult  to 
mandate  specific  detailed  procedures  to  oe  followed  in 
executing  the  surveillance  program,  with  the  possible  less 
of  management  control  being  most  pronounced  under  this 
scheme. 

ether  variations  might  locate  the  contract  manager 
within  ether  divisions.  For  example,  he  may  report  tc  the 
activity's  Maintenance  Control  Director.  There  may  also  be 
mere  than  one  SCM  manager  for  an  activity.  Finally,  all 
functional  division  heads,  given  the  proper  amount  and  level 
of  training,  could  conceivably  Decoma  Service  Contract 
Managers  . 
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The  Air  Force  utilizes  the  following  CA  cor.~rj.rt 
management  structure.  All  Quality  Assurance  Evaluators  ar  ? 
assigned  to  functional  managers  and  are  assisted  an  their 
duties  by  the  QAE  Program  Coordinator,  an  expert  in  the 
field  of  service  contract  surveillance.  QAE’s  are  always 
designated  in  writing,  and  a  candidate's  final  approval  is 
decided  by  the  installation  commander.  Training  and  surveil¬ 
lance  of  QAE  activities  is  handled  by  the  QA  program  coordi¬ 
nator.  Location  of  the  quality  arsurance  organization  is 
decentralized  pursuant  to  policy  stated  in  AF9  7J-9.  Some 
Air  Force  activities  formerly  utilized  centralized  QAE  orga¬ 
nizations;  the  authors'  conversation  with  Air  Force  sources 
revealed  that  such  organizations  no  longer  exist. 

Nc  Navy  policy  exists  that  mandates  either  decen¬ 
tralized  or  centralized  organizations.  At  larger  Public 
Works  Centers,  trends  seem  to  indicate  that  centralized 
types  of  organizations  manage  the  administration  of  service 
contracts.  Within  Public  Worics  Departments,  either  type  of 
organizaticn  may  be  found.  Based  on  the  small  sampling 
taken,  larger  Public  Works  Departments  prooably  locate  all 
QAE  assets  and  the  Service  Contract  Manager  within  the 
Maintenance  Control  rivision. 

Centralization  of  QAE  assets  is  desirable  for 
several  reasons.  One  is  that  effective  contract  management 
control  must  be  established.  Inspectors  placed  under  func¬ 
tional  managers  may  have  little  or  no  allegiance  to  main¬ 
taining  the  integrity  of  the  service  contract  process  and 
may  net  be  able  to  conform  with  standards  set  by  the  5CM. 
There  are  rc  guarantees  that  Quality  Assurance  Evaluators 
will  be  dedicated  strictly  to  performance  of  inspection. 
Thus,  overall  integrity  of  the  service  contract  process  may 
be  sacrificed  where  the  surveillance  responsibiiit y  accrues 
to  a  functional  division  head.  Wcrx  ordering  inputs  may  not 
be  kept  distinct  from  work  output  verifzcation  processes  for 
the  contract,  leading  to  possiole  conflicts  of  interest. 


Special  qualities  of  the  Service  Contract  har.aqer 
were  described  earlier.  His  responsibility  for  growing 
numbers  of  A-76  service  contracts  makes  it  necessary  to 
appoint  him  as  a  division  head. 

The  most  important  reason  rot  to  locate  QAE 
personnel  under  functional  management  is  to  preclude  the 
inadvertent  communication  of  in /alia  implied  contractual 
authority  tc  the  service  contractor.  Air  Force  limitations 
on  QAE  rcles  are  spelled  out  as  follows:  [fief.  38] 

a)  QAE's  will  not  clarify,  interpret  cr  infer  legal 
interpretations  of  contract  scope  or  intent; 

b)  QAE's  will  not  give  direction  to  contractor  employees; 

c)  QAE's  will  not  enter  into  unauthorized  contract  agree¬ 
ments  (including  modifications)  ; 

d)  QAE's  will  not  require  work  to  be  done  that  is  not 
specifically  called  fer  in  the  contract;  and 

e)  QAE's  will  not  authorize  expenditures  of  funds. 

These  actions  are  the  responsibility  of  the  cogni¬ 
zant  contracting  agent.  In  a  centralized  staffing  arrange¬ 
ment,  the  SCI1  can  readily  identify  and  ascertain  the  actions 
cf  the  Quality  Assurance  Evaluators  via  daily  inspection 
reports.  The  SCM  will  be  more  successful  an  providing 
training  for  QAE's  and  assisting  in  their  professional  self 
development.  The  Quality  Assurance  Evaluators  can  mere 
readily  avail  themselves  of  contract  administration  assis¬ 
tance  and  cooperate  to  establish  integrity  in  the  CA 
contract  process. 

Functional  managers  may  oppose  this  centralized 
inspection  organization  concept  on  the  grounds  that  they 
will  lose  direct  control  cf  their  operations.  They  tray 
strongly  believe  that  the  quality  assurance  function  is  an 
inherent  responsibility  of  their  organization,  and  that  no 
outside  centralized  department  should  interfere  wit.n  their 
assigned  missions. 


Cne  distinct  benefit  resulting  from  a  centralized 
arrangement  is  easier  tracking  and  budgeting  for  the  full 
costs  of  service  contract  administration.  Such  costs  may  be 
more  difficult  tc  identify  for  decentralized  organizations. 

An  area  of  uncertainty  in  the  use  of  a  centralized 
organization  involves  job  qualifications  for  QAE's  who 
inspect  unique  technical  CA  functions.  In  the  Public  Works 
environment,  this  issue  might  pertain  to  inspection  cf  util¬ 
ities  and  transportation  eguipment  service  contracts.  The 
activity  must  balance  its  choice  between  requirements  for 
specialized  inspection  skills  and  general  knowledge  of  the 
technical  area. 

6 .  GAO  Examines  POD  Management  Control  cf  Service 

Contracts 

Two  recent  General  Accounting  Office  reports  which 
examined  and  criticized  existing  federal  agency  administra¬ 
tion  of  facility  service  contracts  lend  credence  tc  the 
establishment  of  centralized  inspection  organizations. 

A  report  entitled  "Better  Management  Needed  in  pop 
to  Prevent  Fraudulent  and  Erroneous  Contract  Payments  to 
Reduce  Beal  Property  Maintenance  Costs"  of  9  January  1930 
uncovered  several  instances  of  overpayments  to  contractors 
where  work  was  either  not  performed  or  found  to  be  unsatis¬ 
factory.  GAO  pointed  out  a  lack  cf  effective  inspection 
procedures  and  internal  management  ccntrcis.  Inspector 
reports  were  often  erroneous  and  unreliable.  GAO  called  for 
independent  audits  of  each  inspector’s  performance. 
Specific  GAC  criticisms  were:  [Ref.  46] 


Derrcrmance. 


a)  work  performed  was  non  billed  in  accordance 
contract  provisions; 

b)  inferior  work  was  accepted; 


c)  less  expensive  materials  were  substit uted  fcr  these 
specified  in  the  contract;  and 

d)  seme  work  was  paid  for  more  than  once. 

Seme  GAO  reccmaend ations  for  improvements  were  as 
follows;  [Bef.  46] 

a)  ensure  that  activities  provide  sufficient  numbers  of 
adequately  trained  contract  inspectors; 

b)  require  that  routine  independent  tests  of  each  inspec¬ 
tor’s  work  be  made; 

c)  ensure  that  the  proposed  work  is  adequately  planned 
before  contract  award  and  that  contract  specifications 
are  clear  and  appropriate; 

d)  continue  to  devete  a  portion  of  internal  audit  effort 
tc  local  procurement  activities;  and 

a)  require  that  detailed  inspection  records,  including 
measurements  and  calculations,  be  maintained  in 
support  of  contract  payments. 

These  recommendations  might  be  better  attained  with 
a  centralized  CA  contract  managmenent  organization.  The 
National  Aeronautics  and  Space  Administration  was  criticized 
by  GAG  fcr  certain  inadequacies  in  its  contract  management 
procedures  in  a  21  October  1980  report.  Specific  findings 
included:  [Hef.  47] 

a)  a  contractor  was  working  without  approved  work  orders; 

b)  questionable  reimbursements  occurred  for  the  contrac¬ 
tor's  work  ; 

c)  contract  funds  were  increased  before  the  need  was 
justified;  and 

a)  seme  contracting  officers  had  a  general  attitude  that 
small  dollar  value  contracts  were  not  worthy  of 
adequate  attention. 
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Again,  such  problems  might  oe  mitigated  under  a 
centralized  mode  of  contract  surveillance  and  administra¬ 
tion.  Seth  reports  emphasize  the  importance  of  properly 
organizing  and  locating  a  CA  service  contract  administration 
organization.  The  activity  may  obtain  the  right  number  of 
inspectors,  properly  train  them,  and  then  gain  nothing  if  it 
does  net  join  these  professionals  as  a  unified  team  to 
handle  newly  acquired  surveillance  responsib_licies.  The 
Northern  Division  cf  the  Naval  Facilities  Engineering 
Command  (NGHTHDIV)  is  investigating  alternative  structural 
arrangements  (depicted  in  Appendix  s )  and  has  asked  customer 
activities  tc  provide  opinions  pertinent  to  this  organiza¬ 
tional  issue.  £Ref.  45]  As  stated  earlier,  no  mandatory 
NAVFAC  policy  has  been  established.  A  wide  divergence  in 
attitudes  and  motivations  at  various  Navy  activities  may  be 
reason  tc  refrain  fr cm  mandating  a  specific  policy. 
Immediate  dialogue  on  the  issue  at  DOD  policy-making  levels 
may  help  tc  establish  standardized  CA  inspection  organiza¬ 
tion  guidelines. 

Cther  advantages  cf  centralized  inspection  are  mere 
careful  assessments  cf  proposed  changes  to  current  workload 
and  inci  eased  flexibility  to  adapt  to  these.  More  effective 
inspector  autonomy  will  allow  the  QAS  to  set  schedule  prior¬ 
ities  and  choose  appropriate  surveillance  methods. 

In  decentralized  organizations,  inspectors  may  be 
constricted  in  exercising  such  flexibility  and  their  sched¬ 
ules  may  be  based  or.  the  personal  whims  cf  the  functional 
manager.  Cne  primary  objective  in  creating  an  organizational 
format  is  tc  grant  the  inspectors  proper  responsibility  and 
control. 
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7.  A  Summa ra y  cf  Ad  vantages  arid  Dnsadvar.o  apes  of 
Central izat icn 

Eased  on  the  preceding  discussion.  the  comparative 
advantages  and  disadvantages  of  a  centralized  organization 
are  presented: 

a)  Advantages: 

i)  More  effective  and  reliable  management  ccr.trol 
mechanisms  can  be  instituted  by  the  Service 


Contract  Manager 
integrity. 


cor.tr  act 


ii)  Allegiance  of  Quality  Assurance  Evaluators  is 
obtained  in  ensuring  that  services  are  delivered 
as  required  by  the  contract.  No  dilution  of 
inspector  motivation  results  by  placing  the 
inspector  under  functional  area  managers  whc  may 
not  appreciate  the  nuances  of  contract  surveil¬ 
lance. 

iii)  Ultimate  inspection  costs  may  be  less  with 
centralized  organizations. 

iv)  Centralized  divisions  may  promote  a  greater 
sense  of  professionalism  among  Quality  Assurance 
Evaluators.  Statistical  quality  assurance  tech¬ 
niques  and  contract  admioist rat ion  procedures 
are  mastered  by  each  inspector  through  constant 
cooper  at  icn  and  interface  with  the  SCM. 

v)  The  contractor's  interests  are  better  served  by 
a  centralized  organization;  lines  of  authority 
are  more  visible  and  understandable. 

vi)  Cp pc rt un itiss  for  mismanagement  and  cccurrance 
cf  fraud  should  be  lessened. 

vii)  By  creating  a  centralized  organization,  the 
activity  more  readily  assigns  a  proper  priority 
to  management  of  CA  contracts  ar. i  gains  a  mere 


credible  position  when  it  requests  additional 
funding  cr  training  support. 

viii)  Better  opportunities  for  employee  self  develop¬ 
ment  exist.  Training  can  be  easily  secured  and 
managed.  The  SCM  will  be  able  to  tetter  assess 
the  necessary  requirements  for  each  inspector's 
self  development, 
b)  Disadvantages 

i)  Certain  CA  functions  may  require  technical 
expertise  that  is  not  available  in  the  existing 
QAE  resources.  By  not  placing  the  surveillance 
for  these  under  functional  manager  control,  work 
performance  may  be  jeopardized. 

ii)  Functional  manager  objections  must  be  dealt 
with.  These  might  be  a  perceived  lack  of 
control  that  result  from  the  inability  to  deal 
with  contractors  in  a  face-to-face  relationship. 
They  could  also  feel  that  centralized  inspector 
organizations  will  be  largely  insensitive  to 
special  concerns.  The  end  result  of  such  appre¬ 
hension  may  be  refusal  to  cooperate  with  the 
inspection  organization  and  the  contractor. 

iii)  At  small  naval  activities,  the  cost  cf  a 
centralized  division  may  exceed  the  cost  of 
placing  each  inspector  under  functional  division 
heads. 

Earlier  study  discussion  examined  the  feasibility  of 
a  new  contract  agency  organization  structure  for  the  CA 
service  contracts  that  includes  Navy  intermediate  and  field 
commands.  If  these  activities  embrace  the  concept  of 
centralized  service  contract  divisions,  the  Navy  will  have 
moved  one  step  closer  to  its  creation. 


Use  of  centralized  organizations  will  be  advanta¬ 
geous  should  an  activity  choose  to  use  large  Base  Operating 
Support  type  contract  solicitations.  The  activity  may  also 
desire  tc  fora  guality  assurance  teams  for  specific  contract 
types  which  exhibit  a  combination  of  both  decentralized  and 
centralized  formats. 

C.  JCB  SERIES  DESCRIPTORS  FOR  SERVICE  CONTRACT  MANAGERS 

This  study  has  carefully  described  a  process  for 
building  a  successful  contract  quality  assurance  organiza¬ 
tion  at  the  field  level,  emphasizing  the  formation  of  a 
separate  division  tc  manage  the  administration  of  a  growing 
number  cf  A-76  CA  service  contracts.  This  occurs  after  the 
activity  defines,  estimates,  and  budgets  personnel  require¬ 
ments  tc  carry  out  the  performance  cf  each  contract's 
quality  assurance  plan.  This  process  will  be  an  ongcizc  one 
as  increasing  CA  contracting  occurs. 

During  this  process,  the  activity  needs  to  define  and 
determine  skill  and  knowledge  requirements  for  that  person 
who  may  prove  to  be  most  crucial  m  successfully  manaair.g 
the  practical  i  iple mentati on  of  CA  service  contracts,  the 
Service  Contract  Manager. 

A  description  cf  the  Naval  Facilities  Engineering 
Command  requirements  for  this  person  was  presented  ir.  the 
preceding  section.  Some  additional  requirements  of  the  5CM 
are  discussed. 

A  quality  control  manager  must  be  dole  to  investigate 
and  prepare  plans  to  meet  long  range  quality  control  needs, 
establishing  realistic  objectives.  He  audits  the  effective¬ 
ness  of  in-house  quality  control  procedures  and  spot-checks 
delivered  services  tc  measure  quality.  He  must  be  able  to 
analyze  and  interpret  records  resulting  from  a  quality  assu¬ 
rance  pregram,  and  be  able  to  quickly  dispose  of  matters 
pertaining  to  defective  performance. 
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He  develops  standard  procedures  for  random  sampling 
inspections  and  for  collection,  tabulation,  and  reporting  of 
results  tc  proper  authorities.  Ha  assists  in  surveys  of 
quality  control  techniques  utilized  by  potential  suppliers. 
He  conducts  quality  assurance  training  courses.  He  maintains 
cognizance  cf  all  current  industrial  quality  control  stan¬ 
dards,  and  may  research  and  develop  new  quality  assurance 
techniques.  [Ref.  30] 

Most  Public  Works  Commercial  Activities  fall  into  cr.e  cf 
four  general  categories.  Service  Contract  Managers  should 
have  general  knowledge  of  these  categories  which  ara: 


a)  General  Housekeeping  Services: 

i)  Custodial  services; 

ii)  Refuse  services;  and 

iii)  Grounds  maintenance  services. 

b)  Building  and  Maintenance  Sevices: 

i)  Housing  maintenance  services  ;ar.d 

ii)  Industrial  facili  .  maintenance  services. 

c)  Transportation  Equipment  Operations  ar.d  Maintenance 

services;  and 

d)  Utilities  Operations  and  Maintenance  services. 

When  functions  are  placed  under  cost  study,  these  will 
indicate  the  technical  and  managerial  skills  to  be  required 
cf  Service  Contract  Managers  and  Quality  Assurance 
Evaluators.  Possible  job  series  desnptcrs  for  the  Service 
Contract  Manager  will  be  explored  which  might  match  tech¬ 
nical  and  managerial  capabilities  required  for  this  impor¬ 
tant  activity  position.  Specific  job  series  descriptions  or 
Wage  Grade  classifications  for  Quality  Assurance  Svaluatcrs 
will  ret  be  evaluated  it  the  thesis  due  to  the  diversity  of 
activity  requirements  for  these  positions. 
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The  primary  consideration  in  choosing  Quality  Assurance 
Evaluators  is  deter  lining  if  there  will  be  a  paramount 
requirement  for  actual  trade  or  craft  experience.  if  the 
inspection  duties  requires  that  QAE's  be  able  to  perform  the 
work,  then  a  Wage  Grade  classification  is  required. 
Otherwise,  a  General  Schedule  job  description  will  suffice. 

Civilian  personnel  offices  are  most  qualified  to  deter¬ 
mine  the  classifications  and  grades  for  QAE  billets.  It  is 
imperative  that  the  activity  carefully  define  inspection 
work  requirements  so  that  an  accurate  assessment:  of  specific 
technical  and  managerial  skills  can  be  determined.  These 
will  be  the  basis  cf  the  position  description  that  will  be 
used  to  evaluate  the  qualifications  and  prior  experience  of 
a  prospective  QAE  candidate.  The  Service  contract  Manager 
should  be  involved  in  the  initial  screening  of  such  candi¬ 
dates. 

When  the  Service  Contract  Manager  rcle  was  first  defined 
by  NAVFAC,  it  was  recommended  that  this  position  be  clas- 
sifsd  under  the  GS-1102  series.  Contract  Specialist.  This 
series  primarily  involves  tae  review  and  control  over 
contracts  tc  protect  the  government's  interest  based  upon 
business,  financial,  and  legal  standpoints.  This  series  is 
suitable  if  the  activity  desires  greater  emphais  cr.  main¬ 
taining  the  integrity  cf  its  service  contracts.  It  does  not 
address  technical  knowledge  requirements  which  nay  be  neces¬ 
sary  tc  manage  diverse  types  cf  industrial  functions. 

Another  job  series  that  has  been  suggested  is  the  GS-309 
series.  Construction  Control  Inspector.  Representative 
functions  cr  this  cl assi ficaticn  are  listed:  [Ref.  ad] 

a)  Reviews  of  plans  and  specifications  prior  to  contract 
sc  lie  i  ta  t  i  on  ; 

d)  attends  pre-bid  and  ?  re-constr  notion  conferences; 

c)  supervises  conduct  of  site  surveys; 
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d)  assists  in  development  of  specifications; 

e)  interfaces  with  contractors;  furnishes  requirements 
for  construction  scheduling,  progress  reporting, 
safety  measures,  wage  and  hour  law  requirements; 

f)  observes  and  identifies  all  stages  of  construction, 
and  takes  action  to  correct  problems; 

g)  reviews  contractor  inspection  systems  and  advises  of 
necessary  corrections; 

h)  investigates  and  processes  change  orders;  and 

i)  interfaces  with  local  agencies  and  authorities  during 
construction. 

The  GS-809  series  places  significant  emphasis  cr  tech¬ 
nical  skill  requirements,  seme  emphasis  on  knowledge  of 
contracting  skills,  and  minimal  emphasis  on  managerial  capa¬ 
bilities.  This  position  has  been  used  with  success  for 
supervisory  construction  inspection  positions.  It  would  be 
most  suitable  for  guality  assurance  specialists  or  for 
Service  Contract  Managers  who  will  be  responsible  for 
surveillance  of  maintenance  construction  service  contracts. 

Another  series  is  the  GS-810  Facilities  Engineering 
Manager.  It  covers  duties  in  the  areas  cf  investigations 
and  surveys,  planning  and  design,  construction,  research, 
and  facilities  engineering  management.  Generally  speaking, 
the  GS-810  series  does  not  reflect  a  substantial  requirement 
for  directive  or  supervisory  control  and  may  not  be  a  most 
suitable  choice.  It  also  lacks  a  requirement  for  general 
knowledge  of  contract  administration  procedures.  [Ref.  49] 

Another  alternative  is  the  GS-1640,  series.  Facilities 
Manager,  which  reflects  a  requirement  for  broad  technical 
knowledge  cf  operating  capabilities  and  maintenance  require¬ 
ments  for  an  activitiy's  various  types  of  physical  plant  and 
equipment.  Certain  specific  elements  of  job  performance 
include  maintenance  program  planning,  financial  planning  and 
control,  and  facilities  requirements  planning.  This  posi- 
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tion's  shortcoming  is  that  it  omits  a  raquir ament 
general  knowledge  of  contract  procedures.  [Ref.  50] 

All  series  discussed  thus  far  lack  a  requirement 
knowledge  in  one  area  that  is  important  fcr  the  future 
development  cf  CA  service  contracting,  knowledge  of  statis¬ 
tical  quality  assurance  techniques.  This  has  net  been 
mentioned  in  the  preceding  jcb  series  although  it  is  essen¬ 
tial  if  new  sampling  inspection  systems  recommended  by  A-76 
policy  are  to  be  implemented. 

A  job  series  that  might  best  integrate  the  areas  of 
quality  assurance,  contract  administration  knowledge,  and 
limited  technical  and  managerial  expertise  is  the  GS-1S10 
series.  Quality  Assurance  Inspector.  It  addresses  the 
design  and  administration  of  quality  assurance  systems  that 
involve  monitoring,  controlling  and  maintaining  quality  and 
reliability  for  delivered  goods  and  services.  Specific  job 
elements  include: 

a)  Review  of  the  contractor's  performance, 

b)  review  and  acceptance  of  contractor  quality  control 
systems, 

c)  inspection  of  delivered  services  to  verify  a  contrac¬ 
tor's  stated  quality  cf  services, 

d)  use  cf  random  sampling  techniques  in  service  contract 
surveillance, 

e)  summarization  and  analysis  of  results  of  inspections 
of  services,  ar.d 

f)  resolution  of  contractor  quality  problems. 

The  Air  Force  has  utilized  the  GS-1910  series  for  its 
Quality  Assurance  Pregram  Coordinator  positions  at  the  field 
activity  level.  The  coordinator  conducts  training  of  all 
functional  quality  assurance  evaluators  and  also  assists  in 
the  statement  of  work  and  .quality  assurance  plan  development 
processes.  He  conducts  audits  of  ongoing  quality  assurance 


and  quality  control  procedures  for  all  Air  Force  field 
activity  contracts.  He  may  participate  in  progress  meetings 
that  are  held  with  contractors.  Working  in  conjunction  with 
cognizant  contract  administrators  as  well  as  the  functional 
area’s  Quality  Assurance  Evaluator,  he  is  a  vital  part  of 
the  activity's  service  contract  program.  The  Navy  service 
Contract  Manager  would  perform  essentially  the  same  roles, 
except  that  this  role  is  expanded  to  include  direct  supervi¬ 
sion  of  QAE's. 

Use  of  the  GS-1910  series  is  recommended  as  the  most 
suitable  position  description  for  Service  Contract  Managers. 
Additional  training  in  contract  administration  procedures 
may  te  required,  but  this  can  be  easily  accomodated  by 
sanding  the  SCM  to  a  contracting  school  (such  as  the  Civil 
Engineer  Corps  Officers  School  at  Port  Hueneme,  California.) 
This  position  is  operational  in  nature,  and  requires  a 
working  knowledge  of  various  statistical  quality  assurance 
techniques.  It  is  unlikely  any  of  the  previously  mentioned 
job  series  will  enable  the  activity  to  secure  a  manager  with 
this  specialized  background.  An  industrial  engineering  back¬ 
ground  for  prospective  SCM  candidates  is  also  recommended. 

The  rest  of  the  inspection  organization  can  supplement 
and  augment  the  general  knowledge  held  by  the  Service 
Contract  Manager.  Quality  Assurance  Evaluatcrs  should 
possess  special  skills  in  carpentry,  utility  equipment, 
transportation  vehicle  equipment  and  other  specialized  skill 
requirements.  Based  on  preceding  discussions  of  organiza¬ 
tional  issues,  the  hiring  of  contract  specialists  may  be 
necessitated.  Such  skill  and  technical  knowledge  require¬ 
ments  will  be  influenced  by  the  size  of  a  naval  activity, 
its  complexity,  and  the  number  of  service  contracts  that  are 
being  performed.  It  may  be  necessary  to  hire  statistical 
specialists  at  either  intermediate  or  top  level  commands  so 
that  ongoing  review  and  improvement  of  statistical  surveil¬ 
lance  methods  can  be  carried  cut. 


Activities  must  accurately  determine  the  qualifications 
cf  the  Sevice  Contract  Manager,  especially  if  they  elect  to 
form  a  centralized  service  contract  management  organization. 
The  ability  to  successfully  execute  Commercial  Activities 
will  te  tied  to  the  skills  of  the  SCM  and  his  organization. 


0.  SOMMABY 

This  chapter  has  examined  several  issues  that  relate  to 
the  creation  of  an  inspection  organization  for  these  service 
contracts  that  result  from  OMB  Circular  A-76  implementation. 
The  Navy  field  activity  that  receives  services  has  the  major 
responsibility  fer  ensuring  proper  contract  administration 
by  providing  on-site  surveillance  resources. 

The  activity  must  fellow  a  sequence  of  actions  that  lead 
to  the  creation  of  an  onsite  inspection  organization.  It 
must  first  estimate  hew  many  inspectors  it  will  need,  making 
plans  tc  hire  additional  inspectors  or  utilize  existing 
personnel  resources. 

It  must  ensure  that  a  training  program  is  established 
which  educates  the  inspectors  in  quality  assurance  proce¬ 
dures  and  furthers  their  knowledge  in  contract  administra¬ 
tion  principles.  It  must  secure  funding  for  this  effort. 

The  activity  must  choose  the  organization  format  that 
best  suits  its  surveillance  needs.  Although  research 
suggests  that  a  centralized  format  is  preferable,  the 
activity  may  elect  tc  place  inspectors  under  functional 


managers. 


Either  system  must  offer  suff 


t  offer  sufficient  management 


control  to  ensure  that  contractual  integrity  is  established. 

Finally,  based  upon  the  type  of  contracts  to  be  awarded, 
an  activity  must  define  knowledge  and  skill  requirements  for 
the  duality  Assurance  Evaluators  and  Service  Contract 
Managers.  The  SCM  position  is  vitally  important,  and  a  most 
capable  individual  with  training  in  statistical  quality 
control  techniques  is  required. 
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VII.  BECCBHBNDATICNS  FOB  I8PBOVEHENTS  IN  CA  CONTRACT 

CON VBBSIOHS 

A.  IHPBCVEHENTS  IN  STATISTICAL  QUALITY  ASSURANCE 

It  is  evident  that  quality  assurance  techniques  are 
useful  and  beneficial  in  the  CA  service  contract  process, 
promoting  fairness  and  equity  to  all  concerned  parties  when 
used  properly.  These  techniques  should  ultimately  reduce 
contract  costs  to  the  government  and  contractors,  and  lead 
to  mere  reliable  assessment  and  documentation  of  service 
contract  quality.  Statistical  Quality  Control  (SQC)  appli¬ 
cations  will  be  limited  only  by  the  creativity  and  imagina¬ 
tion  cf  those  who  use  them. 

Accordingly,  the  following  recommendations  for  the 
quality  assurance  process  are  offered. A  formal  working  panel 
of  DOD  officials  should  be  organized  to  review  the  different 
statistical  quality  assurance  methods  that  are  either  being 
utilized  or  being  studied.  Accurate,  reliable  information 
that  examines  the  merits  and  weaknesses  of  acceptance  and 
estimation  sampling  processes  should  be  made  available  to 
this  panel.  Perhaps  an  independent,  unbiased,  quality 
control  expert  should  be  included  on  this  panel  to  ensure 
that  each  member  properly  understands  the  validity  and 
application  of  statistical  quality  assurance  to  CA  service 
contracts.  The  authors  believe  that  any  confusion  among 
various  DOD  CA  management  officials  concerning  SQC  techni¬ 
ques  can  be  eliminated  by  properly  and  accurately  stating 
which  cf  these  can  be  used  in  ongoing  A-76  contracts. 

A  ccst-tenefit  and  risk  analysis  may  be  conducted  to 
examine  which  of  the  two  sampling  processes  are  mere  appro¬ 
priate  for  CA  quality  assurance.  Fixed  budgetary  funding 


101 


amounts  fcr  CA  contract  administration  may  have  significant 
influence  in  the  determination  of  which  standard  system 
should  he  adopted. 

B.  IHPBC  VEMENTS  IB  ADHI BIS TEBIIG  EXTRAPOLATED  DEDUCTIONS 

A  unifcra  policy  should  also  be  adopted  for  the  extrapo¬ 
lated  deductions  process.  As  was  observed  earlier,  the  Air 
Force  does  net  deduct  for  defects  when  these  are  less  than 
the  reject  number  at  the  specified  acceptable  quality  level 
(AQL)  .  New  NAVFAC  policy,  if  adopted,  would  call  for  deduc¬ 
tions  for  all  observed  defects  and  liquidated  damages  set  as 
a  percentage  of  the  value  of  each  observed  defective  work 
item.  Current  Air  Fcrce  and  proposed  NAVFAC  policy  would 
implement  extrapolated  deductions  from  samples  to  lets  when 
specified  AQL' s  are  exceeded. 

The  authors  believe  that  the  NAVFAC  policy  should  be 
adopted.  It  is  more  realistic,  providing  more  enforcement 
power  in  the  long  range  evolution  of  CA  service  contracting. 
It  may  also  be  more  defensible  in  subsequent  GAO  and  ether 
DOD  audits  of  each  agency*  s  CA  contract  administration 
efforts.  The  previously  mentioned  DOD  A-76  workina  panel 
that  studies  sampling  techniques  should  be  tasked  to  resolve 
this  divergence  in  administration  policy. 

The  authors  recommend  an  immediate  resolution  of  this 
issue  tc  allow  the  presentation  to  its  contractor  community 
cf  one  unifcra  DCD  CA  policy  concerning  contract  deductions 
and  inspection.  Failure  to  do  so  will  hinder  optimal  perfor¬ 
mance  cf  government  surveillance,  and  possibly  cause  it  to 
become  ineffective. 
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C.  IHEBC  VEMENTS  IN  ESTIflAT ING  INSPECTION  RESOURCES 

The  first  major  issue  that  was  covered  in  this  topic 
area  was  the  proper  estimation  of  contract  administration 
resources,  particularly  the  determination  of  the  required 
number  cf  Quality  Assurance  Evaluators.  If  a  pclicv  is 
adopted  that  attempts  to  correlate  the  number  of  required 
inspectors  with  the  number  of  contracts,  their  dollar  value, 
or  the  number  of  positions  being  converted,  then  a  compre¬ 
hensive  study  should  be  undertaken  of  A- 76  resource  expendi¬ 
tures  by  all  federal  agencies  and  activities  to  facilitate 
the  development  of  an  estimating  model.  All  facets  cf  CA 
service  ccntract  administration  resources  consumed  for  the 
past  five  to  ten  years  will  need  to  be  carefully  examined; 
such  a  large  inclusive  data  base  will  be  the  most  reliable 
means  of  building  this  model;  however,  the  cost  and  time 
requirements  for  its  development  may  be  prohibitive. 

The  authors  recommend  a  much  simpler  and  more  accurate 
approach  similar  to  that  which  was  developed  by  the  Atlantic 
Division,  Naval  Facilities  Engineering  Command.  This  model, 
when  used  properly,  should  provide  a  relatively  unbiased  and 
unconstrained  estimate  of  resource  requirements.  If  this 
technique  is  used,  sincere  attempts  must  be  made  by  all 
federal  agencies  to  procure  and  place  the  required  inspec¬ 
tion  resources  at  each  field  activity.  A  careful  integra¬ 
tion  of  A-7 6  resource  planning  activities  in  each  agency's 
annual  budget  process  will  be  required. 

If  suboptimization  or  satisficing  occur  in  this  process, 
then  the  first,  more  detailed,  model  should  be  developed 
since  it  will  probably  be  mere  indicative  of  actual  inspec¬ 
tion  requirements. 
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D.  ENHANCEMENT  OF  CQSBENT  QAE  TRAINING 

During  the  course  of  the  study  research,  the  authors 
observed  different  training  programs  for  Quality  Assurance 
Evaluators.  In  general,  these  were  sufficient  in 'providing 
initial  exposure  to  rudiments  of  contract  law,  sampling 
techniques,  and  ether  skill  areas  required  of  QAE's.  These 
basic  training  courses  should  be  enhanced  by  instituting 
more  ongoing  activity  training  programs  which  either  reit¬ 
erate  or  augment  the  fundamental  precepts  of  CA  service 
contract  administration  and,  in  turn,  lead  to  a  more  profes¬ 
sional  QAE  staff. 

The  Air  Force  has  addressed  this  need  in  designing  its 
QAE  training  programs.  The  QA  Program  Coordinator  maintains 
a  continuing  dialogue  with  each  QAE  during  the  performance 
periods  of  service  contracts,  with  contract  administrators 
also  offering  their  advice  and  assistance. 

The  Navy  has  adopted  the  practice  of  offering  regional¬ 
ized  training  for  QAE*  s  at  either  Engineering  Field 
Divisions,  large  Public  Works  Departments,  or  Public  Works 
Centers.  QAE  candidates  travel  to  these  sites  and  spend  a 
week  in  training.  Few  ongoing  training  programs  at  the 
activity  level  were  observed. 

The  authors  recommend  that  existing  agency  efforts  in 
the  training  phase  of  CA  implementation  be  continued. 
Improvements  should  be  effected  in  creating  more  ongoing 
activity  training  programs,  perhaps  adopting  a  programmed 
learning  type  of  instruction  technique.  Training  in  statis¬ 
tical  quality  assurance  can  be  improved  further,  with  a 
greater  emphasis  placed  on  basic  statistical  theory  and 
application  . 

Funding  support  for  QAE  training  is  crucial  to  enable 
the  guality  assurance  programs  fer  CA  service  contracts  to 
be  successful.  Haphazard  and  poorly  planned  funding  will 
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jeopardize  the  enrichment  of  skills  of  each  Q AS  and  possibly 
detract  from  the  successful  evolution  of  the  A-76  program. 

£.  FAC I LIT ATING  CA  AD  BIN I ST EAT I ON  CONTROL  THROOGH 

CENTS  ALIZATI  ON 

A  large  portion  of  the  study  addressed  the  creation  of 
centralized  CA  contract  administration  organizations.  It  is 
realized  that  this  organizational  form  will  not  be  appro¬ 
priate  for  all  DOD  agencies  and  activities  given  a  wife 
diversity  in  size  and  mission  requirements;  many  activities 
may  be  reluctant  to  embrace  this  format  ever,  where  it  is 
proven  to  be  feasible  and  appropriate.  Earlier  discussion, 
however,  indicated  that  the  advantages  of  centralization 
outweigh  the  disadvantages. 

This  structure  also  has  the  advantage  of  Gaining  immed¬ 
iate  management  control  and  integrity  of  ca  contract  admin¬ 
istration  in  the  current  developmental  stages  cf  A-76 
implementation.  Congressional  oversight  and  public  opinion 
will  be  very  critical  of  cost  overruns  and  contract  irregu¬ 
larities,  especially  if  a  large  number  of  displaced  federal 
employees  vcice  their  objections.  Decentralized  control  is 
not  guaranteed  to  result  in  such  abuses,  but  it  would  invite 
their  occurence  more  readily  than  will  centralized  inspec¬ 
tion  organziaoions. Centralized  inspection  will  more  readily 
facilitate  coordinated  inspector  efforts  than  decentralized 
formats.  The  credibility  cf  inspection  efforts  may  be 
enhanced  mere  with  centralized  organizations. 

Additional  reasons  for  supporting  creation  of  central¬ 
ized  organizations  are  that  full  costs  of  inspection  may  be 
reduced,  training  programs  may  be  more  effectively  adminis¬ 
tered,  and  lines  of  communication  from  government  represen¬ 
tatives  tc  contract  administrators  will  be  made  more  clear. 
The  authors  recommend  that  the  centralized  organization 
structure  be  adopted. 
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F.  SEBVICE  CON  TRACT  MANAGE R  QUALIFICATIONS 


The  Service  Contract  Manager  will  be  a  key  activity 
individual  whether  centralized  or  decentralized  organiza¬ 
tions  are  chosen.  He  should  be  well  versed  in  statistical 
sampling  and  have  a  broad  general  oackgrcund  in  contract 
administration  principles,  as  well  as  having  some  knowledge 
cf  the  functional  areas  to  be  contracted  out.  The  authors 


recommend  use  of  the  GS-1910  series,  as  it  offers  t  most 
versitality  and  flexibility  in  overseeing  CA  vice 
contract  admin istr at icn.  Other  job  descriptions  wer-  xam- 
ined,  and  in  fact,  seme  of  these  have  actually  been  u  .  ed 
by  various  Navy  activities.  For  example,  the  GS-1102  series 


could  be  utilized  despite  its  lack  of  technical  knowledge 
requirements.  The  overrriding  necessity  for  knowledge  of  SQC 
techniques  and  their  applications  should  lead  to  uniform 
acceptance  cf  the  GS-1910  series  as  a  CA  contract  adminis¬ 
tration  manager  standard. 

G.  FOTOHE  CA  QOALIT1  ASSURANCE  IMPROVEMENTS 

In  its  Command  Management  Guidance  for  fiscal  years  199u 
to  1990,  NAVFAC  headquarters  stresses  the  attainment  of 
"most  efficient  organization"  (MEO)  structure  by  all  of  its 
activities.  This  thesis  examined  initiatives  in  the  develop¬ 
ment  cf  quality  assurance  programs  which  should  lead  to  such 
cost  economies  while  achieving  required  levels  of  service. 
Ihese  should  also  improve  the  professionalism  exhibited  by 
federal  agencies  in  executing  further  Commercial  Activities 
contracts. 

Comprehensive  planning  efforts  should  be  initiated  at 
all  Navy  organization  levels  to  develop  an  effective  A-76 
implementation  strategy,  and  integrate  contract  design  with 
quality  assurance.  Other  Navy  Systems  Commands  besides 
NAVFAC  are  responsible  for  a  substantial  number  of 
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Commercial  Activities  subject  to  033  Circular  A-78.  These 
commands  should  follow  NAVFAC's  lead  in  executing  their  own 
CA  programs. 

Continued  pursuit  of  the  A-76  policy  will  change  the 
character  of  mission  performance.  DOD  agencies  will  need  to 
carefully  plan  for  their  administration  of  service  contract 
programs  tc  preclude  deterioration  in  the  quality  of  the 
services  they  receive. 
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APPENDIX  A 

LEGISLATIVE  PROVISIONS 


The  following  are  excerpts  from  two  Public  Laws  that 
establish  congressional  policy  and  recurring  restrictions 
concerning  the  conversion  of  DOD  In-house  activities  to 
contract  performance. 


Department  of  Defense  Appro priation  Authorization  Act .  1975 . 
Public  Law  93-  36  5.  August  5  ,  1 974 


SEC.  502. 
Department  o 
manpower  tha 
and  other 
Therefore , 
ticn  regues 
manpower  ocl 
particular, 
one  form  cf 
private  cent 
nob.  A  rul 
form  of  man 
annual  manp 
required  by 
Code. 


It  is 
f  Defense 
t  is  ccns 
needs  o 
in  develop 
ts  to  tn 
icies,  th 
consider 
manpower  t 
ract)  fo 
1  justifi 
power  to  a 
ower  regu 
section  13 


the  sense  of  Congre 
shall  use  the  least  c 
istent  with  military 
f  the  Department 
ing  the  annual  manpow 
e  Conaress  and  ir. 
a  Secretary  of  Defen 
the  advantages  of  co 
o  another  (military, 
r  the  performance  of 
cation  of  any  ccnvers 
nether  shall  be  cent 
irements  report  to 
8 (c)  (3)  of  title  1  0 , 


ss  that  the 
ostly  form  of 
requirements 
of  Defense, 
sr  aathcriza- 
carrying  cut 
sa  shall,  m 
nverting  from 
civilian,  or 
a  specified 
ion  from  one 
ained  m  the 
the  Conaress 
United  States 


Department  cf  Defense  Autho rizatior.  Act ,  1981.  Public  Law 

96-342.  September  9,  1980 


SEC.  502. 

a)  Nc  commercial  or  industrial  type  function  cf  the 
Department  of  Defense  that  on  October  1,  1980,  is 
being  performed  by  Department  of  Defense  personnel 
may  be  converted  tc  performance  by  a*  private 
ccntractcr-- 


persennei 


tc  circumvent  any  civilian  personnel 
ceiling;  or 

unless  the  Secretary  of  Defense  provides  tc 
the  Congress  in  a  timely  manner  — 


[i] 


1. 


notification  of  any  decision  to  srudy 
such  commercial  or  industrial  type  func¬ 
tion  for  possible  performance  by  a 
private  contractor ; 

2.  a  detailed  summary  of  a  comparison  of 
the  cost  o.  performance  of  such  function 
by  Department  of  Defense  personnel  and 
by  private  contractor  which  demonstrates 
that  the  performance  of  such  function  by 
a  private  contractor  will  result  in  a 
cost  savings  to  the  Government  over  the 
life  of  the  contract  and  a  certification 
that  the  entire  cost  comparison  is  avai¬ 
lable; 

3.  a  certification  that  the  Government 

calculation  for  the  cost  of  performance 
of  such  function  by  Department  of 

Defense  personnel  is  Dased  cn  at  esti¬ 
mate  of  the  most  efficient  and  cost 
effective  organization  for  performance 
of  such  function  by  Department  cf 

Defense  personnel;  and 

4.  a  report.  to  be  submitted  with  the 

certification  required  by  subparagraph 
3.  showing —  tne  potential  economic 
effect  on  employees  affected,  and  the 
potential  economic  effect  on  the  local 
community  and  Federal  Government  if  mere 
than  50  employees  are  involved,  of 

contracting  for  performance  cf  such 
function;  the  effect  of  contract ira  fer 
performance  of  such  function  on'  the 
military  mission  of  such  function;  and 
the  amount  of  the  bid  accepted  for  the 
performance  of  such  function  by  the 
private  contractor  whose  bid  is  accepted 
and  the  cost  of  performance  cf  such 
function  by  Department  of  Defense 
personnel,  toaether  with  costs  and 
expenditures  which  the  Government  will 
incur  because  of  the  contract. 

t)  If,  after  completion  of  the  studies  required  for 
completion  or  the  certification  and  report 
required  by  subparagraphs  3  and  4  of  subsection 
aii) ,  a  decision  is  maae  to  convert  to  contractor 
performance,  the  Secretary  of  Defense  shall  notify 
Congress  of  such  decision. 

c)  Ihe  Secretary  of  Defense  shall  submit  a  written 

report  to  the  Congress  by  February  1  cf  each 
fiscal  year  describing  the  extent  to  which  commer¬ 
cial  ana  industrial  type  functions  were  performed 
by  Department  of  Defense  contractors  during  the 
preceding  fiscal  year.  The  Secretary  shall 

include  m  each  such  report  an  estimate  cf  the 
percentage  of  commercial  and  industrial  type  func¬ 
tions  of  the  Department  of  Defense  that  will  be 
performed  by  Department  of  Defense  personnel,  and 
the  percentage  cf  such  functions  that  will  be 
performed  by  private  contractors,  during  the 
fiscal  year  during  which  the  report  is  submitted. 

d)  This  section  shall  take  effect  on  October  1  ,  1930. 
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APPENDIX  B 

EXCERPTS  FROfl  HIL-STD-105D  AND  BIL-HHDBK-53-1 A 

In  this  appendix,  information  contained  in  both  Military 
Standard  105D  and  Military  Handbook  53-1A  is  presented.  The 
text  cf  MIL-STD-105D  is  given  in  pages  111  through  120.  Key 
issues  to  be  considered  in  the  use  of  the  standard  are  the 
lot  size,  the  inspection  intensity,  and  the  desired  accep¬ 
table  quality  level  (AQL) .  A  determination  of  these  varia¬ 
bles  results  in  a  sampling  size  and  accept/reject  numbers. 
Operating  characteristic  (QC)  curves  for  various  sampling 
plans  are  also  presented  in  pages  121  through  127.  These 
represent  the  protection  that  is  offered  to  bcth  the 
contractor  and  the  government,  being  a  function  of  the 
percent  cf  defectives  that  are  found  in  samples. 

Pages  128  through  131  are  selected  pages  from 
MIL-HNDBK-53-1 A  which  provide  more  detailed  explanations  of 
the  operating  characteristic  curves.  Page  132  is  an  excerpt 
from  a  randcm  number  table.  Finally,  pages  133  and  134  are  a 
listing  of  hey  activities  to  be  followed  in  implementing 
MIL-STD-105D. 


3 


SAMPLING  PROCEDURES  AND  TABLES 
FOR  INSPECTION  BY  ATTRIBUTES 


1.  SCOPE 


1.1  PURPOSE.  This  publication  estab¬ 
lishes  sampling  plans  and  procedures  for 
inspection  by  attributes.  When  specified  by 
the  responsible  authority,  this  publication 
shall  be  referenced  in  the  specification,  con¬ 
tract,  inspection  instructions,  or  other  docu¬ 
ments  and  the  provisions  set  forth  herein 
shall  govern.  The  "responsible  authority" 
shall  be  designated  in  one  of  the  above 
documents. 

1.2  APPLICATION.  Sampling  plans  des¬ 
ignated  in  this  publication  are  applicable,  but 
not  limited,  to  inspection  of  the  following: 

a.  End  items. 

b.  Components  and  raw  materials. 

c.  Operations. 

d.  Materials  in  process. 

e.  Supplies  in  storage. 

f.  Maintenance  operations. 

g.  Data  or  records. 

h.  Administrative  procedures. 

These  plans  are  intended  primarily  to  be 
used  for  a  continuing  series  of  lots  or  batches. 


The  plans  may  also  be  used  for  the  inspection 
of  isolated  lots  or  batches,  but,  in  this  latter 
case,  the  user  is  cautioned  to  consult  the 
operating  characteristic  curves  to  find  a  plan 
which  will  yield  the  desired  protection  (see 
11.6). 

1 .3  INSPECTION.  Inspection  is  the  proc¬ 
ess  of  measuring,  examining,  testing,  or 
otherwise  comparing  the  unit  of  product  (see 
1.5)  with  the  requirements. 

1.4  INSPECTION  BY  ATTRIBUTES.  In¬ 
spection  by  attributes  is  inspection  whereby 
either  the  unit  of  product  is  classified  simply 
as  defective  or  nondefective,  or  the  number 
of  defects  in  the  unit  of  product  is  counted, 
with  respect  to  a  given  requirement  or  set 
of  requirements. 

1.5  UNIT  OF  PRODUCT.  The  unit  of 
product  is  the  thing  inspected  in  order  to 
determine  its  classification  as  defective  or 
nondefective  or  to  count  the  number  of  de¬ 
fects.  It  may  be  a  single  article,  a  pair,  a  set, 
a  length,  an  area,  an  operation,  a  volume,  a 
component  of  an  end  product,  or  the  end 
product  itself.  The  unit  of  product  may  or 
may  not  be  the  same  as  the  unit  of  purchase, 
supply,  production,  or  shipment. 


2.  CLASSIFICATION  OF  DEFECTS  AND  DEFECTIVES 


2.1  METHOD  OF  CLASSIFYING  DEFECTS. 

A  classification  of  defects  is  the  enumeration 
of  possible  defects  of  the  unit  of  product 
classified  according  to  their  seriousness.  A 
defect  is  any  nonconformance  of  the  unit  of 
product  with  specified  requirements.  Defects 
will  normally  be  grouped  into  one  or  more 
of  the  following  classes;  however,  defects 
may  be  grouped  into  other  classes,  or  into 
subclasses  within  these  classes. 

2.1.1  CRITICAL  DEFECT.  A  critical  de¬ 
fect  is  a  defect  tha .  judgment  and  experience 
indicate  is  likely  to  result  in  hazardous  or 
unsafe  conditions  for  individuals  using, 
maintaining,  or  depending  upon  the  product; 
or  a  defect  that  judgment  and  experience 
indicate  is  likely  to  prevent  performance  of 
the  tactical  function  of  a  major  end  item  such 
as  a  ship,  aircraft,  tank,  missile  or  space 
vehicle.  NOTE;  For  a  special  provision  re¬ 
lating  to  critical  defects,  see  6.3. 

2.1.2  MAJOR  OEFECT.  A  major  defect 
is  a  defect,  other  than  critical,  that  is  likely 
to  result  in  failure,  or  to  reduce  materially 
the  usability  of  the  unit  of  product  for  its 
intended  purpose. 


2.1.3  MINOR  DEFECT.  A  minor  defect 
is  a  defect  that  is  not  likely  to  reduce  ma¬ 
terially  the  usability  of  the  unit  of  product 
for  its  intended  purpose,  or  is  a  departure 
from  established  standards  having  little  bear¬ 
ing  on  the  effective  use  or  operation  of  the 
unit. 

2.2  METHOD  OF  CLASSIFYING  DEFEC¬ 
TIVES.  A  defective  is  a  unit  of  product  which 
contains  one  or  more  defects.  Defectives  will 
usually  be  classified  as  follows: 

2.2.1  CRITICAL  DEFECTIVE.  A  critical 
defective  contains  one  or  more  critical  de¬ 
fects  and  may  also  contain  major  and  or 
minor  defects.  NOTE;  For  a  special  provi¬ 
sion  relating  to  critical  defectives,  see  6.3. 

2.2.2  MAJOR  DEFECTIVE.  A  major  de¬ 
fective  contains  one  or  more  major  detects, 
and  may  also  contain  minor  defects  but  con¬ 
tains  no  critical  defect. 

2.2.3  MINOR  DEFECTIVE.  A  minor  de¬ 
fective  contains  one  or  more  minor  defects 
but  contains  no  critical  or  major  defect. 


3.  PERCENT  DEFECTIVE  AND  DEFECTS  PER  HUNDRED  UNITS 


3.1  EXPRESSION  OF  NONCONFORM¬ 
ANCE.  The  extent  of  nonconformance  of 
product  shall  be  expressed  either  in  terms 
of  percent  defective  or  in  terms  of  defects  per 
hundred  units. 

3.2  PERCENT  DEFECTIVE.  The  percent 
defective  of  any  given  quantity  of  units  of 
product  is  one  hunderd  times  the  number  of 
defective  units  of  product  contained  therein 
divided  by  the  total  number  of  units  of  prod¬ 
uct,  i.e.: 


Percent  defective 


Number  of  defectives 
Number  ol  units  inspected 


100 


3.3  DEFECTS  PER  HUNDRED  UNITS.  The 

number  of  defects  per  hundred  units  of  any 
given  quantity  of  units  of  product  is  one 
hundred  times  the  number  of  defects  con¬ 
tained  therein  ( one  or  more  defects  being 
possible  in  any  unit  of  product)  divided  by 
the  total  number  of  units  of  product,  i.e.: 

Defects  per  Number  of  defects  •  ^ 

hundred  units  ~  Numher  of  units  inspected 


4.  ACCEPTABLE  QUALITY  LEVEL  (AQL) 


4.1  USE.  The  AQL,  together  with  the 
Sample*  Size  Code  Letter,  is  used  for  index¬ 
ing  the  sampling  plans  provided  herein. 

4.2  DEFINITION.  The  AQL  is  the  max¬ 
imum  percent  defective  (or  the  maximum 
number  of  defects  per  hundred  units)  that, 
for  purposes  of  sampling  inspection,  can  be 
considered  satisfactory  as  a  process  average 
(see  11.2). 

4.3  NOTE  ON  THE  MEANING  OF  AQL. 

When  a  consumer  designates  some  specific 
value  of  AQL  for  a  certain  defect  or  group 
of  defects,  he  indicates  to  the  supplier  that 
his  (the  consumer's)  acceptance  sampling 
plan  will  accept  the  great  majority  of  the  lots 
or  batches  that  the  supplier  submits,  pro¬ 
vided  the  process  average  level  of  percent 
defective  (or  defects  per  hundred  units)  in 
these  lots  or  batches  be  no  greater  than  the 
designated  value  of  AQL.  Thus,  the  AQL 
is  a  designated  value  of  percent  defective  (or 
defects  per  hundred  units)  that  the  consumer 
indicates  will  be  accepted  most  of  the  time 
by  the  acceptance  sampling  procedure  to  be 
used.  The  sampling  plans  provided  herein 
are  so  arranged  that  the  probability  of  ac¬ 
ceptance  at  the  designated  AQL  value  de¬ 
pends  uprn  the  sample  size,  being  generally 
higher  for  large  samples  than  for  small  ones, 
for  a  given  AQL.  The  AQL  alone  does  not 

S.  SUBMISSION 

5.1  LOT  OR  BATCH.  The  term  lot  or 
batch  shall  mean  “inspection  lot"  or  “inspec¬ 
tion  batch,"  i.e.,  a  collection  of  units  of  prod¬ 
uct  from  which  a  sample  is  to  be  drawn  and 
inspected  to  determine  conformance  with  the 
acceptability  criteria,  and  may  differ  from  a 
collection  of  units  designated  as  a  lot  or  batch 


describe  the  protection  to  the  consumer  for 
individual  lots  or  batches  but  more  directly 
relates  to  what  might  be  expected  from  a 
series  of  lots  or  batches,  provided  the  steps 
indicated  in  this  publication  are  taken.  It  is 
necessary  to  refer  to  the  operating  character¬ 
istic  curve  of  the  plan,  to  determine  what 
protection  the  consumer  will  have. 

4.4  LIMITATION.  The  designation  of  an 
AQL  shall  not  imply  that  the  supplier  has 
the  right  to  supply  knowingly  any  defective 
unit  of  product. 

4.5  SPECIFYING  AQLs.  The  AQL  to  be 

used  will  be  designated  in  the  contract  or  by 
the  responsible  authority.  Different  AQLs 
may  be  designated  for  groups  of  defects  con¬ 
sidered  collectively,  or  for  individual  defects. 
An  AQL  for  a  group  of  defects  may  be  des¬ 
ignated  in  addition  to  AQLs  for  individual 
defects,  or  subgroups,  within  that  group. 
AQL  values  of  10  0  or  less  may  be  expressed 
either  in  percent  defective  or  in  defects  per 
hundred  units;  those  over  10  0  shall  be  ex¬ 
pressed  in  defects  per  hundred  units  only. 

4.6  PREFERRED  AQLs.  The  values  of 
AQLs  given  in  these  tables  are  known  as 
preferred  AQLs.  If.  for  any  product,  an  AQL 
be  designated  other  than  a  preferred  AQL, 
these  tables  are  not  applicable. 

OF  PRODUCT 

for  other  purposes  (eg,  production,  ship¬ 
ment,  etc.). 

5.2  FORMATION  OF  LOTS  OR  BATCHES. 

The  product  shall  be  assembled  into  identi¬ 
fiable  lots,  sublots,  batches,  or  in  such  other 
manner  as  may  be  prescribed  (see  5  4).  Each 
lot  or  batch  shall,  as  far  as  is  practicable, 


.Id 


5.  SUBMISSION  OF  PRODUCT  (Continued) 


consist  of  units  of  product  of  a  single  type, 
grade,  class.  size,  and  composition,  manu¬ 
factured  under  essentially  the  same  condi¬ 
tions,  and  at  essentially  the  same  time. 

5.3  LOT  OR  BATCH  SIZE.  The  lot  or 
batch  size  is  the  number  of  units  of  product 
in  a  lot  or  batch. 

5.4  PRESENTATION  OF  LOTS  OR 
BATCHES.  The  formation  of  the  lots  or 

6.  ACCEPTANCE 

41  ACCEPTABILITY  OF  LOTS  OR 
BATCHES.  Acceptability  of  a  lot  or  batch 
will  be  determined  by  the  use  of  a  sampling 
plan  or  plans  associated  with  the  designated 
AQL  or  AQLs. 

6.2  DEFECTIVE  UNITS.  The  right  is  re¬ 
served  to  reject  any  unit  of  product  found 
defective  during  inspection  whether  that 
unit  of  product  forms  part  of  a  sample  or 
not,  and  whether  the  lot  or  batch  as  a  whole 
is  accepted  or  rejected.  Rejected  units  may 
be  repaired  or  corrected  and  resubmitted  for 
inspection  with  the  approval  of,  and  in  the 
manner  specified  by,  the  responsible  au¬ 
thority. 

6.3  SPECIAL  RESERVATION  FOR  CRITI¬ 
CAL  DEFECTS.  The  supplier  may  be  required 
at  the  discretion  of  the  responsible  authority 
to  inspect  every  unit  of  the  lot  or  batch  for 


batches,  lot  or  batch  size,  and  the  manner 
in  which  each  lot  or  batch  is  to  be  presented 
and  identified  by  the  supplier  shall  be  des¬ 
ignated  or  approved  by  the  responsible  au¬ 
thority.  As  necessary,  the  supplier  shall 
provide  adequate  and  suitable  storage  space 
for  each  lot  or  batch,  equipment  needed  for 
proper  identification  and  presentation,  and 
personnel  for  all  handling  of  product  re¬ 
quired  for  drawing  of  samples. 

AND  REJECTION 

critical  defects.  The  right  is  reserved  to  ir. 
spect  every  unit  submitted  by  the  supplier  for 
critical  defects,  and  to  reject  the  lot  or  batch 
immediately,  when  a  critical  defect  is  found. 
The  right  is  reserved  also  to  sample,  for  crit¬ 
ical  defects,  every  lot  or  batch  submitted  by 
the  supplier  and  to  reject  any  lot  or  batch 
if  a  sample  drawn  therefrom  is  found  to  con¬ 
tain  one  or  more  critical  defects. 

6.4  RESUBMITTED  LOTS  OR  BATCHES. 

Lots  or  batches  found  unacceptable  shall  be 
resubmitted  for  reinspection  only  after  all 
units  are  re-examined  or  retested  and  all  de¬ 
fective  units  are  removed  or  defects  cor¬ 
rected.  The  responsible  authority  shall  deter¬ 
mine  whether  normal  or  tightened  inspection 
shall  be  used,  and  whether  reinspection  shall 
include  all  types  or  classes  of  defects  or  for 
the  particular  types  or  classes  of  defects 
which  caused  initial  rejection. 


7.  DRAWING  O?’  SAMPLES 


7.1  SAMPLE.  A  sample  consists  of  one 
or  more  units  of  product  drawn  from  a  lot  or 
batch,  the  units  of  the  sample  being  selected 
at  random  without  regard  to  their  quality. 
The  number  of  units  of  product  in  the  sample 
is  the  sample  size. 


7.2  REPRESENTATIVE  SAMPLING.  When 
appropriate,  the  number  of  units  in  the  sam¬ 
ple  shall  be  selected  in  proportion  to  the  size 
of  sublots  or  subbatches,  or  parts  of  the  lot  or 
batch,  identified  by  some  rational  criterion. 


7.  DRAWING  OF  SAMPLES  (Continued) 


When  representative  sampling  is  used,  the 
units  from  each  part  of  the  lot  or  batch  shall 
be  selected  at  random. 

7.3  TIME  OF  SAMPLING.  Samples  may 
be  drawn  after  all  the  units  comprising  the 
lot  or  batch  have  been  assembled,  or  sam¬ 


ples  may  be  drawn  during  assembly  o'  the 
lot  or  batch. 

7.4  DOUBLE  OR  MULTIPLE  SAMPLING. 

When  double  or  multiple  sampling  is  to  be 
used,  each  sample  shall  be  selected  over  the 
entire  lot  or  batch. 


8.  NORMAL,  TIGHTENED  AND  REDUCED  INSPECTION 


8.1  INITIATION  OF  INSPECTION.  Nor¬ 
mal  inspection  will  be  used  at  the  start  of 
inspection  unless  otherwise  directed  by  the 
responsible  authority. 

8.2  CONTINUATION  OF  INSPECTION. 

Normal,  tightened  or  reduced  inspection 
shall  continue  unchanged  for  each  class  of 
defects  or  defectives  on  successive  lots  or 
batchs  except  where  the  switching  proce¬ 
dures  given  below  require  change.  The 
switching  procedures  given  below  require  a 
change.  The  switching  procedures  shall  be 
applied  to  each  class  of  defects  or  defectives, 
independently. 

8.3  SWITCHING  PROCEDURES. 

8.3.1  NORMAL  TO  TIGHTENED.  When 
normal  inspection  is  in  effect,  tightened  in¬ 
spection  shall  be  instituted  when  2  out  of  S 
consecutive  lots  or  batches  have  been  re¬ 
jected  on  original  inspection  (i.e..  ignoring 
resubmitted  lots  or  batches  for  this  proce¬ 
dure). 

8.3.2  TIGHTENED  TO  NORMAL.  When 
tightened  inspection  is  in  effect,  normal  in¬ 
spection  shall  be  instituted  when  5  consecu¬ 
tive  lots  or  batches  have  been  considered 
acceptable  on  original  inspection. 

8.3.3  NORMAL  TO  REDUCED.  When 
normal  inspection  is  in  effect,  reduced  inspec¬ 
tion  shall  be  instituted  providing  that  all  of 
the  following  conditions  are  satisfied: 


a.  The  preceding  10  lots  or  batches  (or 
more,  as  indicated  by  the  note  to  Table  VIII) 
have  been  on  normal  inspection  and  none 
has  been  rejected  on  original  inspection:  and 

b.  The  total  number  of  defectives  (or  de¬ 
fects)  in  the  samples  from  the  preceding  10 
lots  or  batches  (or  such  other  number  as  was 
used  for  condition  "a"  above)  is  equal  to  or 
less  than  the  applicable  number  given  in 
Table  VIII.  If  double  or  multiple  sampling 
i„  in  use,  all  samples  inspected  should  be  in¬ 
cluded.  not  ''first’'  samples  only;  and 

c.  Production  is  at  a  steady  rate;  and 

d.  Reduced  inspection  is  considered  de¬ 
sirable  by  the  responsible  authority. 

8.3.4  REDUCED  TO  NORMAL.  When  re¬ 
duced  inspection  is  in  effect,  normal  inspec¬ 
tion  shall  be  instituted  if  any  of  the  following 
occur  on  original  inspection: 

a.  A  lot  or  batch  is  rejected;  or 

b.  A  lot  or  batch  is  considered  acceptable 
under  the  procedures  of  10  1.4;  or 

c.  Production  becomes  irregular  or  de¬ 
layed;  or 

d  Other  conditions  warrant  that  normal 
inspection  shah  be  instituted. 

8.4  DISCONTINUATION  OF  INSPECTION. 

In  tHe  evght  that  10  consecutive  lots  or 
batches  remain  on  tightened  inspection  (or 
such  other  number  as  may  be  designated  by 
the  responsible  authority),  inspection  under 
the  provisions  of  this  document  should  be 
discontinued  pending  action  to  improve  the 
quality  of  submitted  material. 


9.  SAMPLING  PLANS 


9.1  SAMPLING  PLAN.  A  sampling  plan 
indicates  the  number  of  units  of  product 
from  each  lot  or  batch  which  are  to  be  in¬ 
spected  (sample  size  or  series  of  sample 
sizes)  and  the  criteria  for  determining  the 
acceptability  of  the  lot  or  batch  (acceptance 
and  rejection  numbers). 

9.2  INSPECTION  LEVEL  The  inspection 
level  determines  the  relationship  between 
the  lot  or  batch  size  and  the  sample  size.  The 
inspection  level  to  be  used  for  any  particular 
requirement  will  be  prescribed  by  the  re¬ 
sponsible  authority.  Three  inspection  levels: 
I,  II,  and  III,  are  given  in  Table  I  for  general 
uie.  Unless  otherwise  specified,  Inspection 
Level  II  will  be  used.  However,  Inspection 
Level  I  may  be  specified  when  less  discrimi¬ 
nation  is  needed,  or  Level  III  may  be  speci¬ 
fied  for  greater  discrimination.  Four  addi¬ 
tional  special  levels:  S-l,  S-2,  S-3  and  S— 4. 
are  given  in  the  same  table  and  may  be  used 
where  relatively  small  sample  sizes  are  neces¬ 
sary  and  large  sampling  risks  can  or  must  be 
tolerated. 

NOTE:  In  the  designation  of  inspection 
levels  S-l  to  S— 4,  care  must  be  exercised  to 
avoid  AQLs  inconsistent  with  these  inspec¬ 
tion  levels. 

9.3  CODE  LETTERS.  Sample  sizes  are 
designated  by  code  letters.  Table  I  shall  be 
used  to  find  the  applicable  code  letter  for  the 
particular  lot  or  batch  size  and  the  prescribed 
inspection  level. 

9.4  OBTAINING  SAMPLING  PUN.  The 
AQL  and  the  code  letter  shall  be  used  to  ob¬ 


tain  the  sampling  plan  from  Tables  II,  III  or 
IV.  When  no  sampling  plan  is  available  for  a 
given  combination  of  AQL  and  code  letter, 
the  tables  direct  the  user  to  a  different  letter. 
The  sample  size  to  be  used  is  given  by  the 
new  code  letter  not  by  the  original  letter.  If 
this  procedure  leads  to  different  sample  sizes 
for  different  classes  of  defects,  the  code  letter 
corresponding  to  the  largest  sample  size  de¬ 
rived  may  be  used  for  all  classes  of  defects 
when  designated  or  approved  by  the  respon¬ 
sible  authority.  As  an  alternative  to  a  single 
sampling  plan  with  an  acceptance  number 
of  0,  the  plan  with  an  acceptance  number  of  1 
with  its  correspondingly  larger  sample  size 
*ora  designated  AQL  ( where  available ) ,  may 
be  used  when  designated  or  approved  by  the 
responsible  authority. 

9.5  TYPES  OF  SAMPLING  PUNS.  Three 
types  of  sampling  plans:  Single,  Double  and 
Multiple,  are  given  in  Tables  II,  III  and  IV, 
respectively.  When  several  types  of  plans  are 
available  for  a  given  AQL  and  code  letter, 
any  one  may  be  used.  A  decision  as  to  type 
of  plan,  either  single,  double,  or  multiple, 
when  available  for  a  given  AQL  and  code 
letter,  will  usually  be  based  upon  the  com¬ 
parison  between  the  administrative  difficulty 
and  the  average  sample  sizes  of  the  available 
plans.  The  average  sample  size  of  multiple 
plans  is  less  than  for  double  (except  in  the 
case  corresponding  to  single  acceptance  num¬ 
ber  1 )  and  both  of  these  are  always  less  than 
a  single  sample  size.  Usually  thfe  administra¬ 
tive  difficulty  for  single  sampling  and  the 
cost  per  unit  of  the  sample  are  less  than -for 
double  or  multiple. 


10.  DETERMINATION  OF  ACCEPTABILITY 


10.1  PERCENT  DEFECTIVE  INSPECTION. 
To  determine  acceptability  of  a  lot  or  batch 
under  percent  defective  inspection,  the  ap¬ 
plicable  sampling  plan  shall  be  used  in 
accordance  with  10.1.1.  10.1.2,  10.1.3,  10.1.4, 
and  10.1.5. 

10.1.1  SINGLE  SAMPLING  PLAN.  The 
number  of  sample  units  inspected  shall  be 
equal  to  the  sample  size  given  by  the  plan. 
If  the  number  of  defectives  found  in  the 
sample  is  equal  to  or  less  than  the  acceptance 
number,  the  lot  or  batch  shall  be  considered 
acceptable.  If  the  number  of  defectives  is 
equal  to  or  greater  than  the  rejection  num¬ 
ber,  the  lot  or  batch  shall  be  rejected. 

10.1.2  DOUBLE  SAMPLING  PLAN.  The 
number  of  sample  units  inspected  shall  be 
equal  to  the  first  sample  size  given  by  the 
plan.  If  the  number  of  defectives  found  in 
the  first  sample  is  equal  to  or  less  than  the 
first  acceptance  number,  the  lot  or  batch 
shall  be  considered  acceptable.  If  the  num¬ 
ber  of  defectives  found  in  the  first  sample  is 
equal  to  or  greater  than  the  first  rejection 
number,  the  lot  or  batch  shall  be  rejected. 
If  the  number  of  defectives  found  in  the  first 
sample  is  between  the  first  acceptance  and 
rejection  numbers,  a  second  sample  of  the 
size  given  by  the  plan  shall  be  inspected.  The 


number  of  defectives  found  in  the  first  and 
second  samples  shall  be  accumulated.  If  the 
cumulative  number  of  defectives  is  equal  to 
or  less  than  the  second  acceptance  number, 
the  lot  or  batch  shall  be  considered  accept¬ 
able.  If  the  cumulative  number  of  defectives 
is  equal  to  or  greater  than  the  second  rejec¬ 
tion  number,  the  lot  or  batch  shall  be  rejected. 

10.1.3  MULTIPLE  SAMPLE  PLAN.  Under 

multiple  sampling,  the  procedure  shall  be 
similar  to  that  specified  in  10.1.2,  except  that 
the  number  >f  successive  samples  required 
to  reach  a  decision  may  be  more  than  two. 

10.1.4  S  P  E  C  I A  L  PROCEDURE  FOR  RE¬ 
DUCED  INSPECTION.  Under  reduced  in¬ 
spection,  the  sampling  procedure  may  termi¬ 
nate  without  either  acceptance  or  rejection 
criteria  having  been  met.  In  these  circum¬ 
stances,  the  lot  or  batch  will  be  considered 
acceptable,  but  normal  inspection  will  be 
reinstated  starting  with  the  next  lot  or 
batch  (see  8.3.4  (b)). 

10.2  DEFECTS  PER  HUNDRED  UNITS  IN¬ 
SPECTION.  To  determine  the  acceptability 
of  a  lot  or  batch  under  Defects  per  Hundred 
Units  inspection,  the  procedure  specified  for 
Percent  Defective  inspection  above  shall  be 
used,  exceDt  that  the  word  “defects”  «h»!l  be 
substituted  for  ''defectives.” 


11.  SUPPLEMENTARY  INFORMATION 


11.1  OPERATING  CHARACTERISTIC 
CURVES.  The  operating  characteristic  curves 
for  normal  inspection,  shown  in  Table  X 
(pages  30-62),  indicate  the  percentage  of 
lots  or  batches  which  may  be  expected  to  be 
accepted  under  the  various  sampling  plans 
for  a  given  process  quality.  The  curves  shown 
are  for  single  sampling;  curves  for  double 


and  multiple  sampling  are  matched  as  closely 
as  practicable.  The  O.  C  curves  shown  for 
AQLs  greater  than  10.0  are  based  on  the 
Poisson  distribution  and  are  applicable  for 
defects  per  hundred  units  inspection;  those 
for  AQLs  of  10  0  or  less  and  sample  sizes  of 
80  or  less  are  based  on  the  binomial  distri¬ 
bution  and  are  applicable  for  percent  defec- 
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live  inspection;  those  for  AQLs  of  10.0  or 
less  and  sample  sizes  larger  then  80  are  based 
on  the  Poisson  distribution  and  are  applica¬ 
ble  either  for  defects  per  hundred  units  in¬ 
spection,  or  for  percent  defective  inspection 
(the  Poisson  distribution  being  an  adequate 
approximation  to  the  binomial  distribution 
under  these  conditions).  Tabulated  values, 
corresponding  to  selected  values  of  probabil¬ 
ities  of  acceptance  (Pa, in  percent)  arc  given 
for  each  of  the  curves  shown,  and,  in  addi¬ 
tion,  for  tightened  inspection,  and  for  defects 
per  hundred  units  for  aQLs  of  10.0  or  less 
and  sample  sizes  of  80  or  less. 

11.2  PROCESS  AVERAGE.  The  process 
average  is  the  average  percent  defective  or 
average  number  of  defects  per  hundred  units 
(whichever  is  applicable)  of  product  sub¬ 
mitted  by  the  supplier  for  original  inspec¬ 
tion.  Original  inspection  is  the  first  inspec¬ 
tion  of  a  particular  quantity  of  product  as 
distinguished  from  the  inspection  of  product 
which  has  been  resubmitted  after  prior 
rejection. 

11.3  AVERAGE  OUTGOING  QUALITY 
(AOG).  The  AOQ  is  the  average  quality  of 
outgoing  product  including  all  accepted  lots 
or  batches,  plus  all  rejected  lots  or  batches 
after  the  rejected  lots  or  batches  have  been 
effectively  100  percent  inspected  and  all  de¬ 
fectives  replaced  by  nondefectives. 

11.4  AVERAGE  OUTGOING  QUALITY 
LIMIT  (AOQL).  The  AOQL  is  the  maximum 
of  the  AOQs  for  all  possible  incoming  quali¬ 
ties  for  a  given  acceptance  sampling  plan. 
AOQL  values  are  given  in  Table  V-A  for 
each  of  the  single  sampling  plans  for  normal 
inspection  and  in  Table  V-B  for  each  of  the 
single  sampling  plans  for  tightened  inspec¬ 
tion. 


11.5  AVERAGE  SAMPLE  SIZE  CURVES. 

Average  sample  size  curves  for  double  and 
multiple  sampling  are  in  Table  IX.  These 
show  the  average  sample  sizes  which  may  be 
expected  to  occur  under  the  various  sampling 
plans  for  a  given  process  quality.  The  curves 
assume  no  curtailment  of  inspection  and  are 
approximate  to  the  extent  that  they  are 
based  upon  the  Poisson  distribution,  and  that 
the  sample  sizes  for  double  and  multiple 
sampling  are  assumed  to  be  0  631n  and  0  25n 
respectively,  where  n  is  the  equivalent  single 
sample  size. 

11.6  LIMITING  QUALITY  PROTECTION. 

The  sampling  plans  and  associated  proce¬ 
dures  given  in  this  publication  were  designed 
for  use  where  the  units  of  product  are  pro¬ 
duced  in  a  continuing  senes  of  lots  or  batches 
over  a  period  of  time.  However,  if  the  lot 
or  batch  is  of  an  isolated  nature,  it  is  desira¬ 
ble  to  limit  the  selection  of  sampling  plans 
to  those,  associated  with  a  designated  AQL 
value,  that  provide  not  less  than  a  specified 
limiting  quality  protection.  Sampling  plans 
for  this  purpose  can  be  selected  by  choosing 
a  Limiting  Quality  (LQ)  and  a  consumer's 
risk  to  be  associated  with  it.  Tables  VI  and 
VII  give  values  of  LQ  for  the  commonly  used 
consumer’s  risks  of  10  percent  and  5  percent 
respectively.  If  a  different  value  con¬ 
sumer's  risk  is  required,  the  O.C.  curves  and 
their  tabulated  values  may  be  used.  The 
concept  of  LQ  may  also  be  useful  in  specify¬ 
ing  the  AQL  and  Inspection  Levels  for  a 
series  of  lots  or  batches,  thus  fixing  minimum 
sample  size  where  there  is  some  reason  for 
avoiding  (with  more  than  a  given  consumer's 
risk)  more  than  a  limiting  proportion  of  de¬ 
fectives  (or  defects)  in  any  single  lot  or 
batch. 


TABLE  X-H — Tables  for  sample  she  code  letter: 
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Note:  Bmonu.it  distribution  used  tar  percent  detective  compulation;  Poisson  tor  defects  per  hundred  units. 


SAMPLING  PLANS  FOR  SAMPLE  SIZE  COOE  LETTER: 
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Binomial  distribution  used  lor  percent  detective  computation;  Poisson  lor  delects  per  hundred  units. 


10.4.1  Selecting  the  Sampling  Plan.  Two  factors  are  generally  con¬ 
sidered  in  the  selection  of  a  sampling  plan:  (1)  the  consumer  and/ 
or  supplier  risk  factor  and  (2)  the  economic  factor.  The  risk  pat¬ 
tern  of  each  sampling  plan  is  represented  by  the  0C  curve  for  the 
plan.  The  OC  curve  for  each  plan  is  different,  a  property  which 
provides  an  effective  means  for  ascertaining  the  effect  of  changes 
in  sample  size  and  acceptance  number  on  the  acceptance  or  rejection 
of  a  lot.  The  proper  (with  respect  to  risk)  sampling  plan  can  be 
determined  from  studying  the  OC  curve  for  each  plan  under  consider¬ 
ation.  By  studying  the  OC  curves,  it  is  possible  t>  compare  the 
relative  risks  of  two  or  more  sampling  plans  for  a  given  sampling 
situation.  By  virtue  of  the  OC  curve,  sampling  tables  can  be  con¬ 
structed  in  which  risks  of  incorrect  decisions  have  been  determined 
in  advance,  making  it  possible  co  select  plans  which  will  have  risk 
factors  that  are  acceptable  to  both  the  supplier  and  the  consumer. 

The  OC  curve,  then,  can  be  used  for  classifying  sampling  plans  from 
the  standpoint  of  the  protection  afforded  to  the  supplier  (AQL  plans), 
consumer  (LQ  plans),  or  both.  The  economic  factor  must  be  considered 
each  time  a  sampling  plan  is  to  be  selected  and,  of  course,  becomes 
more  and  more  important  as  the  cost  of  testing  goes  up.  This  factor 
becomes  especially  important  when,  because  of  the  high  cost  of  test¬ 
ing,  sample  size  must  be  limited  to  a  degree  which  forces  a  compromise 
of  the  risk  requirements  specified  for  the  sampling  plan.  Another 
approach  to  selecting  sampling  plans  is  used  by  some  organizations 
which  handle  many  types  of  items.  Instead  of  selecting  a  sampling 
plan  on  an  item  by  item  basis  as  the  above  procedure  suggests,  a 
standard  operating  procedure  is  established  whereby  a  particular 

very  stringent  sampling  plan  (probably  acceptance  number  of  zero 
and  large  sample  size,  perhaps  the  entire  population)  is  designated 
to  use  when  inspecting  any  quality  characteristic  that  may  be  a 
critical  defect,  a  second  but  less  stringent  sampling  Flan  is  desig¬ 
nated  to  use  when  inspecting  any  quality  characteristic  or  group  of 
quality  characteristics  that  will  be  at  worst  a  major  defect(s),  and 
a  third  and  still  less  stringent  sampling  plan  is  designated  to  use 
when  inspecting  any  quality  characteristic  or  group  of  quality  char¬ 
acteristics  that  will  be  no  worse  than  a  minor  defect(s). 

10.4.2  E  f  f e c  ts  of  Changes  to  the  Sampling  Plan  on  the  OC  Curve . 

A  sampling  plan  and  its  associated  risks  are  completely  defined  by 
the  lot  size,  sample  size,  and  acceptance  number.  The  lot  size, 
except  in  the  case  of  very  small  lots,  has  relatively  little  import¬ 
ance  in  most  cases  in  determining  the  risks  associated  with  any 
given  sampling  plan.  Thus,  sample  sizes  and  acceptance  numbers  are 
Che  two  Important  factors  which  influence  the  risk  pattern  of  sampling 
plans.  If  the  risks  of  a  tentative  sampling  plan  are  considered  un¬ 
satisfactory,  the  question  which  follows  is:  "What  changes  must  be 
made  to  obtain  the  desired  sampling  protection?"  This  can  be  answered 
by  considering  the  effect  on  the  OC  curve  of  changes  in  the  sampling 
plan.  To  understand  Che  effect  of  such  changes,  a  more  detailed  study 
of  the  OC  curve  (see  Figure  2)  is  appropriate.  From  examination  of 
this  curve  it  is  seen  that  if  lots  to  be  inspected  are  2%  defective, 
approximately  90%  of  the  lots  are  expected  to  be  accepted,  whereas  if 
the  lots  submitted  are  8%  defective,  about  10%  of  the  lots  are 
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expected  to  be  accepted.  If  2%  defective  and  82  defective  represent 
good  and  bad  quality  lots,  respectively,  the  good  lots  will  be  re¬ 
jected  10*  (100  -  90  *  10)  of  the  time  (producer's  risk)  and  bad 
lots  accepted  but  10%  of  the  time  (consumer's  risk).  This  rejection/ 
acceptance  frequency  will  occur  by  chance.  If  this  frequency  is 
Intolerable,  appropriate  changes  to  the  sampling  plan  are  required. 

10.4.3  Changes  in  Sample  Size.  An  increase  in  sample  size  results 
In  a  steepening  of  the  0C  curve,  as  indicated  in  Figure  3.  The 
steeper  the  OC  curve,  the  greater  the  power  of  the  sampling  plan  to 
discriminate  between  "good"  and  "bad"  quality.  Figure  3  clearly 
illustrates  the  effect  that  increasing  sample  size  has  on  taking 
the  OC  curve  "steeper". 

10.4.4  Changes  in  Acceptance  Number.  Figure  4  illustrates  the 
effect  of  changes  in  the  ac cep t ance / r e j ec t ion  numbers  on  the  OC 
curve.  Tn  general,  the  effect  n  f  inc  r  s  ■>' n  g  the  acceptance  number 
is  to  shift  the  location  of  the  entire  0C  curve  to  the  right. 

Changing  the  sampling  plan  in  this  way  generally  increases  the 
probability  of  accepting  a  lot  at  a  given  quality  level. 
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QUALITY  OF  INC0MNG  LOTS  (PERCENT  DEFECTIVE) 


FIGURE  2.  O.C.  CURVE  FOR  A  TYPICAL  SAflPLING  PLAN 


10.4.5  Simultaneous  Change  of  Sample  Size  and  Acceptance  dumber. 

If  it  is  desired  to  have  more  accurate  disposition  of  the  lots  whose 
percent  defective  is  close  to  the  selected  quality  level  (the  AQL 
or  the  LQ  for  example),  Che  sample  size  must  be  increased  to  provide 
more  discrimination.  Also,  the  acceptance  number  must  be  selected 
which  will  yield  the  OC  curve  that  is  properly  located  about  the 
"desired"  quality  level.  Thus,  if  the  degree  of  discrimination  of 
a  given  plan  is  considered  adequate,  but  the  probability  of  accepting 
a  lot  at  a  given  quality  level  is  too  great  (i.e.,  the  plan  is  "too 
loose")  or  too  small  (i.e.,  the  plan  is  "too  tight"),  proper  adjust¬ 
ment  is  made  by  selecting  the  appropriate  acceptance  number.  Usually 
in  practice,  if  a  sampling  plan  is  desired  which  has  certain  desirabl 
risk  characteristics,  both  sample  size  and  acceptance  numbers  must  be 
simultaneously  adjusted  (See  Figure  5).  In  order  to  make  proper 
adjustment,  however,  the  effect  of  each  must  be  understood. 


FIGURE  S.  EFFECT  OF  SIMULTANEOUS  CHANCE  OF  SAMPLE  SIZE 
AND  ACCEPTANCE  NUMBER  OH  O.C.  CURVE 
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SECTION  20: 


EQUENCE  OF  OPERATIONS  IN  USINC  MIL-STD-105 


A  typical  sequence  of  operations  in  using  the  sampling  pro¬ 
cedures  and  tables  for  inspection  by  attributes  of  MIL-STD-105 
is  illustrated  by  Table  C  which  follows.  This  table  assumes  a 
requirement  for  single  sampling. 


TABLE  C :  Sequence  of  Operational  S tops 


Steps 


Explanation 


1.  Determine  lot  size. 


2.  Determine  inspection 
1  eve  1 . 

3.  Determine  sample  size 
code  letter  . 

4.  Determine  sar. tiling  plans. 

5.  Establish  severity  of 
inspection. 

6.  Determine  sample  size 
and  acceptance  number. 


7.  Select  sample. 


8.  Inspect  sample. 


9.  Record  inspection 
results. 


1.  Lot  size  controlled  by  lot 
formation  criteria  contained  in 
procurement  documents.  Otherwise, 
establish  by  agreement  between 
responsible  authority  and  supplier. 

2.  If  the  item  specification  does 
not  give  the  inspection  level,  use 
inspection  level  II. 

3.  Found  in  Table  I,  M1L-STD-105, 
based  on  lot  size  and  inspection 

1  eve  1 . 

4.  Single  sampling  generally  selected. 
Double  or  multiple  sampling  may  be  used. 

5.  Normal  inspection  generally  used  at 
start  of  contract  or  production. 

6.  Assuming  normal  inspection  and  given 
the  specified  AQL  value  and  the  sample 
size  code  letter,  the  sample  size  and 
acceptance  number  are  found  in  Table 
II-A,  MIL-STD-105. 

7.  The  sample,  consisting  of  the  number 
of  units  of  product  as  determined  from 
Table  II-A,  MIL-STD-105,  is  selected  at 
random  from  the  lot.  Additionally,  any 
obvious  defectives  that  have  not  been 
selected  for  the  inspection  sanple  are 
removed  from  the  lot  (but  are  not  in¬ 
cluded  in  the  sample).  (See  para.  14.2) 

8.  The  defectives  (or  defects)  are 
counted.  If  this  count  does  not  exceed 
the  acceptance  number  (Ac),  the  entire 
lot  is  accepted.  If  the  count  equals 
or  exceeds  the  rejection  number,  the 
lot  is  rejected. 

9.  Compute  estimated  process  average 
if  required  by  operating  procedures. 
Maintain  record  of  accept /rtject  deci¬ 
sions  in  order  that  switching  rules 
may  be  followed. 


Steps  Explana t Ion 

10.  Resubmit  lot.  10. If  the  lot  is  not  accepted,  it 

may  be  resubmitted  for  acceptance 
inspection  only  after  all  units  of 
the  lot  are  reinspected  and  all 
defective  units  removed  or  reworked. 

Example  6:  Obtaining  a  Plan.  Suppose  the  AQL  is  1.0,  the  inspection 
level  is  II  and  the  lot  size  is  2,500.  The  first  thing  required  is  the 
sample  size  code  letter  (usually  called  simply  the  code  letter,  for 
short).  For  a  lot  size  of  2,500  and  inspection  level  II,  Table  I 
gives  the  code  letter  as  K.  In  the  appropriate  master  table  (Table 
II-A),  it  is  found  that  the  sample  size  for  single  sampling  is  125. 

AQLs  for  normal  inspection  are  given  along  the  top  of  the  table,  and 
under  the  value  1.0  we  find  the  numbers  3  and  4  given  under  the  heading 
Ac  Re  (which  stand  for  acceptance  uu^lc.'  and  rejection  number,  respect¬ 
ively).  The  sampling  plan  required  is: 

Sample  size  125 

Acceptance  number  3 

Rejection  number  4 

Alternatively,  Table  X-K-2  could  be  used.  Again  the  sample  size 
of  125  is  found;  and  in  the  column  for  AQL  1.0  are  found  the  accep¬ 
tance  and  rejection  numbers  3  and  4  as  before. 

Example  7:  Arrows  in  Tables  II,  III,  and  IV.  Suppose  the  AQL  is  0.40, 
the  Inspection  level  is  I,  and  Che  lot  size  is  230.  Table  I  gives  the 
code  letter  as  E.  Using  Table  II-A,  it  is  found  that  there  is  no  plan 
for  letter  E  and  AQL  0.40  but  a  downward  pointing  arrow  that  directs 
us  to  letter  G  instead,  and  the  required  plan  is: 

Sample  si?  32 

Acceptance  number  0 

Rejection  number  1 

Alternatively,  the  specifying  of  code  letter  E  leads  us,  in  the  ex¬ 
tended  tables,  to  Table  X-E-2.  But  this  page  has  no  column  for 
AQL  0.40.  Instead,  the  symbol  of  and  inverted  triangle  appears 
for  AQLs  less  than  1.0,  This  triangle  refe-s  to  the  footnote  "Use 
next  subsequent  sample  size  code  letter  for  wl ich  acceptance  and  re¬ 
jection  numbers  are  available."  If  the  tri  t.  jie  is  thought  of  as  an 
arrowhead,  it  is  pointing  towards  the  edge  c  the  page  to  be  turned. 
This  leads  to  letter  F  where  again  A0L  0.40  not  given,  and  on  to 
letter  G  to  find  the  same  plan  as  before.  It  is  very  important  to 
remember  that  if  a  triangle  or  series  of  triangles  directs  you  from 
one  page  to  another  of  the  extended  tables,  or  an  arrow  directs  you 
from  one  row  to  another  of  the  master  tables,  the  sample  size  to  be 
used  is  the  one  given  for  the  new  page  or  the  new  row  arrived  at  and 
not  Che  one  given  for  the  original  page  or  row  [9.4).  Where  upward 
pointing  arrows  or  triangles  are  found  the  meaning  is  similar.  The 
triangles  again  point  to  the  edge  of  the  page  to  be  turned. 
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APPENDIX  C 

SXCEBPTS  FROM  OPPP  PAMPHLET  NO.  4 


Twc  chapters  frcm  OFPP  Pamplet  No.  4,  "A  Guide  for 
Writing  and  Administering  Performance  Statements  of  Wor'  for 
Service  Contracts",  are  provided  in  this  appendix.  These  are 
Chapter  4,  The  Surveillance  Plan,  and  Chapter  5,  Doing 
Surveillance.  Chapter  4  addresses  quality  assurance  plan 
design  and  illustrates  the  role  that  MIL-STD-105D  plays  in 
the  sampling  process.  Chapter  5  elanorates  on  the  adminis¬ 
trative  procedures  necessary  in  conducting  inspection.  It 
should  be  noted  that  the  OFPP  approach  is  equivalent  tc  Air 
Force  inspection  procedures  given  in  AFR  400-26.  Finally,  a 
Performance  Requirement  Summary  <P3S)  form  is  shown. 


CHAPTER  4 


THE  SURVEILLANCE  PLAN 


4-1.  Basic  Approach.  This  chapter  describes  the  major 
contents  of  a  surveillance  plan.  There  are  three  key  ideas 
that  are  the  basis  for  a  surveillance  plan. 

a.  Management  By  Exception.  Quality  assurance  relates 
to  the  output  service  provided  by  the  contractor.  As  pointed 
out  earlier,  the  output  service  can  result  either  from  a 
contractor-developed  procedure  or  from  an  government  specified 
procedure.  When  the  procedure  is  specified  by  the  government, 
compliance  with  the  procedure  is  the  desired  output  service. 

(1)  When  the  output  is  based  on  a  contractor 
developed  procedure,  the  procedures  are  only  looked  at  on  a 
by-exception  basis;  that  is,  satisfactory  performance  of  the 
output  service  as  specified  in  the  contract  normally 
indicates  that  the  contractor  is  using  satisfactory 
procedures.  The  government  should  be  concerned  only  when 
services  are  not  adequately  performed. 

(2)  In  this  case,  the  inspector  looks  beyond  the 
level  of  services  provided  only  to  determine  if  the  problem 
is  caused  by  the  government  or  the  contractor.  If  government 
provided  items  to  the  contractor's  operation  (such  as, 
parts,  equipment,  or  facilities)  are  at  fault,  action  must  be 
taken  through  government  channels  to  correct  the  problem.  No 
action  will  be  required  of  the  contractor.  When  the  problem 
is  the  contractor's  fault,  the  contractor  is  told  to  take 
corrective  action. 

b.  Performance  Indicator.  The  level  of  contractor 
provided  services  is  monitored  by  checking  the  performance 
values  in  the  statement  of  work  (SOW).  As  described  in 
chapter  2,  a  performance  value  is  a  feature  of  the  service 
that  can  be  measured  by  a  number.  For  example,  two 
important  performance  values  in  vehicle  maintenance  and 
vehicle  operations  are  vehicle  out-of-commission  (VOC)  rate 
and  taxi  response  time. 

c.  Problem  Location.  When  performance  values  show 
that  the  service  is  not  adequately  performed,  the  QAE  uses 
decision  tables  to  locate  the  problem.  The  tables  provide  a 
logical  sequence  to  find  the  problem  cause.  Basically,  they 
are  a  set  of  pointers  which  should  find  the  problem's  source 
in  a  step-by-step  fashion.  The  construction  and  use  of 
decision  tables  are  described  in  paragraph  4-4b. 


Figure  3-1.  Performance  Requirements  Summary. 


4-2. ‘  Surveillance  Information  Sources.  There  are  four 
principal  sources  of  information  for  surveillance:  management 
information  systems,  random  sampling,  checklists,  and  formal 
customer  complaints.  The  following  sections  describe  the 
information  sources  in  detail. 

a.  Management  Information  Systems.  In  a  few  instances, 
an  existing  management  information  system  (MIS)  may  be 
available  as  a  means  of  surveillance.  When  a  MIS  is 
available,  as  in  the  case  of  the  Air  Force's  vehicle  integrated 
management  system  (VIMS)  in  the  vehicle  maintenance  area,  it 
can  collect  information  on  performance  values  which  can 

be  used  instead  of  random  sampling  data. 

(1)  Management  information  systems  usually  collect 
information  for  100  percent  of  the  activities  for  a 
specified  period  of  time.  This  information  can  be  compared 
to  a  contract  standard.  On  the  basis  of  this  comparison, 
performance  can  be  judged  and  the  performance  for  the 
specified  period  accepted  or  rejected. 

(2)  For  example,  the  vehicle  out-of-commission  (VOC) 
rate  is  computed  every  month  by  the  VIMS.  A  simple 
comparison  of  the  VOC  rate  with  the  maximum  acceptable  VOC 

in  the  SOW  explains  a  great  deal  about  the  level  of 
maintenance  service  supporting  the  base  vehicles  and 
organizations. 

(3)  By  way  of  caution,  however,  one  must  check  the 
data  input  into  a  MIS  if  the  system  is  maintained  by  the 
contractor.  If  one  is  going  to  use  a  MIS  to  check  the 
contractor,  make  sure  the  MIS  contains  reliable  data. 

b.  Random  Sampling.  The  most  frequently  used  way  of 
service  contract  surveillance  is  random  sampling.  Services 
are  sampled  by  the  QAE  to  determine  if  the  contractor's 
level  of  performance  is  acceptable.  Acceptance  sampling  is 
done,  basically,  to  determine  a  course  of  action:  that  is, 
whether  to  accept  or  reject  the  contractor's  level  of 
performance  during  a  given  period  of  time.  If  it  rejects 
performance,  certain  actions  are  started.  If  it  accepts 
performance,  no  action  is  taken. 

(1)  The  basis  for  doing  random  sampling  is 
MIL-STD-105D,  Sampling  Procedures  and  Tables  for  Inspection 
by  Attributes  which  is  widely  understood  and  used  by  both  the 
government  and  contractors.  It  is  based  on  the  concept  of  an 
attribute.  An  attribute  is  a  feature  of  a  service  which 
either  does,  or  does  not,  match  a  standard  (for  example,  a 
taxi  is  on  time  or  it  is  not  on  time)  . 
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(2)  When  sampling  by  attributes,  a  certain  number 
of  observations  will  match  the  standards  and  the  remaining 
number  will  not  match.  Therefore,  attribute  sampling  is 
useful  for  describing  how  a  job  is  done,  in  terms  of  defects 
per  hundred  observations,  or  percent  defective.  Using  this 
concept,  sampling  for  a  performance  indicator  can  be 
developed  by  proceeding  through  a  number  of  formal  steps 
based  on  MIL-STD-105D.  The  use  of  these  concepts  is 
described  in  paragraph  4-3,  Sampling  Plan. 

c.  Surveillance  Checklists.  Checklists  are  also  used 
to  check  contract  performance.  They  must  be  used  sparingly, 
however.  The  use  of  the  MIS  and  random  sampling  are  preferred 
information  sources.  Checklists  help  in  surveillance  of 
contract  requirements  that  happen  infrequently.  (For  example, 
if  a  contractor  is  required  to  perform  a  service  once  a 
month,  this  service  would  be  included  on  a  checklist.)  Any 
service  that  is  not  provided  on  a  daily  basis  should  be 
considered  for  inclusion  on  a  checklist  unless  a  MIS  can  be 
used  to  determine  the  quality  of  the  service. 

d.  Formal  Customer  Complaints.  Even  the  best 
surveillance  plan  will  not  allow  the  QAE  to  check  all  aspects 
of  the  contractor's  performance. 

(1)  Formal  customer  complaints  are  a  means  of 
documenting  certain  kinds  of  service  problems.  The  way  to 
get  and  document  customer  complaints  needs  to  be  carefully 
planned  by  the  persons  checking  the  service  contract. 

(2)  Customer  complaints  are  not  truly  random.  They 
are  seldom  used  to  reject  a  service  or  deduct  money  from  the 
contractor. 

(3)  When  random  sampling  is  the  chosen  method  of 
surveilance,  a  customer  complaint  cannot  be  used  to  satisfy  a 
random  observation.  However,  it  can  be  used  as  further 
evidence  of  unsatisfactory  performance  if  random  sampling 
ahows  that  the  specific  service  is  unsatisfactory.  These 
complaints  can  be  used  to  decide  if  action  other  than  a 
deduction  should  be  taken. 

(a)  Getting  Customer  Complaints.  An 
aggressive  customer  complaint  program,  once  established, 
needs  to  be  briefed  to  every  organization  that  receives  the 
contractor's  services.  An  operating  instruction  should  be 
given  to  each  organization  outlining  the  customer  complaint 
program,  the  format  and  the  content  of  a  formal  customer 
complaint,  and  the  action  which  can  be  expected  from  those 
assigned  to  watching  and  managing  the  service  contract. 


123 


(b)  Documenting  the  Customer  Complaint. 
Normally,  each  customer  complaint  is  brought,  either  in 
person  or  by  telephone,  to  the  person  checking  contract 
performance.  Enter  information  about  the  complaint  into  a 
Customer  Complaint  Record,  similar  to  the  sample  shown  in 
figure  4-1.  The  record  contains  the  following  information: 

2  Date  and  time  of  complaint. 

2  Source  of  complaint  -  organization  and 

individual. 


description) . 
services. 


2  Nature  of  complaint  (narrative 

£  Contract  reference  of  complaint  related 

j>  Valid  complaint  (Yes  or  No)  . 

2  Date  contractor  informed  of  complaint. 


2  Action  taken  by  contractor. 

8  Signature  of  the  person  receiving  and 
validating  the  complaint. 


4-3.  Sampling  Plan.  As  a  rule,  a  plan  contains  information 
on  the  acceptable  quality  level,  lot  size,  sample  size,  and 
rejection  level.  It  states  the  number  of  units  from  each  lot 
to  be  inspected  (that  is,  the  sample  size).  It  also  states 
the  criteria  for  determining  the  acceptability  of  the  lot 
(acceptance  and  rejection  numbers).  This  information  is  used 
to  build  the  sampling  guide  which  are  the  major  products  in  a 
surveillance  plan  for  a  service  contract. 


a.  Beginning  the  Plan.  To  begin  building  a  sampling 
plan,  go  to  the  Performance  Requirements  Summary  developed 
during  the  "Write  Statement  of  Work"  step,  chapter  3,  figure 


3-1. 


(1)  This  chart  contains  the  required  services,  the 
standards,  and  acceptable  quality  levels.  At  this  time 
decide  how  the  services  will  be  checked  (what  information 
source  or  method  of  surveillance  will  be  used). 

(2)  Show  these  decisions  on  the  chart.  For  each 
service  where  random  sampling  is  used,  complete  the  steps 
described  below. 
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CUSTOMER  COMPLAINT  RECORD 


Oate  and  Time  of  Complaint:  21  Jan  1979  /  : 1 OOS 


Source  of  Complaint 

Organization:  382  Bomb  Wing/LGC 

Individual:  Capt  John  Murry 


Nature  of  Complaint:  Called  wrecker  and  it  did  not  arrive  until 
3  hours  after  the  request. 


Contract  Reference:  F-5,  para  5. 1.1. 2. 5  and  Performance 
Requirements  Summary. 


Validation:  Contract  requires  a  1  hour  response  time.  Complaint 
is  valid. 


Date  and  Time  Contractor  Informed  of  Complaint:  21  Jan  79/-.1030 


Action  Taken  by  Contractor: 

Contractor  had  a  person  out  sick  and  did  not  have  a  back  up  driver. 
He  has  now  developed  a  roster  of  back  up  drivers  who  can  operate  a 
wrecker. 


Received  and  Validated  8y:  H.  Smyth/QAE 


Figure  4-1.  Customer  Complaint  Record. 


b.  Deciding  on  the  Acceptable  Quality  Level  ( AQL) .  The 

AQL  is  the  highest  number  of  defects  per  hundred,  highest 
percent  defective  or  highest  number  of  defects  that  can  be 
allowed  for  any  service  performance  indicator.  There  are 
only  a  limited  number  of  AQLs  listed  in  MIL-STD-105D  but,  in 
virtually  all  cases,  one  will  be  close  enough  to  control  the 
contractor's  level  of  service. 

(1)  The  first  step  in  designing  a  sampling  plan 
under  MIL-STD-1Q5D  is  the  selection  of  a  realistic  AQL.  No 
service  can  be  perfectly  performed.  The  AQLs  placed  on  the 
Performance  Requirements  Summary  in  figure  3-1,  must  be 
adjusted  at  this  time. 

(2)  Find  the  closest  AQL  from  figure  4-2  and  use  it 
to  replace  the  original  AQL  on  the  Performance  Requirement 
Summary.  For  example,  the  AQL  for  taxi  service  might  have 
been  5  percent.  This  would  be  changed  to  4  percent  or  6.5 
percent  since  5  percent  does  not  appear  in  the  figure. 

c.  Determining  the  Lot  Size.  To  determine  the  sample 
size,  the  lot  size  must  be  known.  The  lot  is  how  often  the 
contractor  provides  the  service  in  a  period  of  time. 

(1)  To  determine  the  lot  size,  estimate  (or  count) 
the  frequency  of  the  service  to  be  sampled,  during  the  period 
it  is  to  be  sampled.  Thus,  if  scheduled  bus  service 
timeliness  is  the  service  being  sampled,  and  a  sample  is 
taken  each  month,  the  lot  size  is  the  number  of  times  that 
are  available  during  the  month  to  observe  bus  timeliness. 

In  this  case,  it  would  be  the  number  of  times  the  buses  go 
around  all  the  routes  each  day,  multiplied  by  the  number  of 
days  in  each  month  on  which  the  bus  routes  operate. 

(2)  In  the  case  of  workorders,  the  monthly  lot  size 
can  be  estimated  from  historical  information  on  file.  The 
projected  workload  data  gathered  in  chapter  2  is  used  to  help 
determine  lot  sizes. 

d.  Determining  the  Sample  Size.  Use  figure  4-3  to 
identify  an  appropriate  sample  size  for  a  given  lot  size. 

(1)  Use  the  normal  sample  size  column  unless  there 
is  a  limited  number  of  QAEs  or  unless  the  cost  of  an 
inspection  suggests  the  use  of  the  medium  or  small  sample 
size  column. 

(2)  Use  the  medium  or  small  sample  size,  if 
inspections  for  a  particular  service  are  lengthy  or  hinder 
the  contractor's  ability  to  provide  service  to  customers. 
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Allowable  Acceptable  Quality  Levels 


0.010  X 

1.0  X 

0.015  x 

1.5  X 

0.025  X 

2.5  X 

0.040  X 

4.0  X 

0.065  X 

6.5  X 

0.10  X 

10.  X 

0.15  X 

15.  X 

0.25  X 

M 

• 

u> 

CM 

0.40  X 

40.  X 

0.65  X 

65.  X 

Figure  4-2.  List  of  MIL-ST0-1C5D  Acceptable  Quality  Levels. 


e.  Selecting  the  Rejection  Level.  Use  MIL-STD-105D 
to  identify  the  acceptance  and  rejection  level  for  the 
sample  size  (see  figure  4-4).  To  use  the  figure,  begin  with 
the  known  values  for  the  AQL  and  the  sample  size. 

(1)  Find  the  selected  sample  size  (in  the  sample 
size  column)  and  read  across  that  line  to  the  column  for  the 
selected  AQL.  At  that  point  there  will  either  be  two  numbers 
or  an  arrow  pointing  up  or  down. 

(2)  If  there  is  an  arrow,  follow  the  direction  of 
the  arrow  until  it  leads  to  a  pair  of  numbers.  Of  the  two 
numbers  at  the  intersection  or  at  the  end  of  the  arrow,  the 
number  on  the  left  (Ac  or  accept)  indicates  the  maximum 
number  of  defects  which  can  occur  in  a  sample  and  still 
permit  the  total  group  or  lot  to  be  judged  acceptable. 

(3)  when  there  is  no  accept  of  reject  number  for  a 
given  sample  size  and  AQL,  following  the  arrow  will  also 
cause  a  change  in  sample  size.  For  example,  with  an  AQL  of 
1.5  and  a  sample  size  of  20,  the  sample  size  would  become  32. 

(4)  The  number  on  the  right  (Re  or  reject) 
indicates  the  minimum  number  of  defects  that  occur  in  a 
sample  which  causes  the  total  group  or  lot  to  be  judged 
unacceptable.  For  example,  suppose  the  sample  size  is 
determined  to  be  32  and  the  AQL  has  been  set  at  6.5  defects 
per  hundred.  Find  the  number  32  in  the  sample  size  column 
and  read  across  that  line  until  the  AQL  column  for  6.5  has 
been  reached.  The  two  numbers  at  that  intersection  are  5  and 
6. 

(a)  In  other  words,  the  number  on  the  left,  5, 
is  the  number  of  defects  which  can  be  found  in  a  sample  and 
still  permit  acceptance  of  the  lot. 

(b)  The  number  6,  to  the  right  of  5,  is  the 
smallest  number  of  defects  needed  to  declare  the  lot 
unacceptable  and  subject  to  further  check,  using  the 
decision  tables. 

4-4.  Developing  the  Sampling  Method.  The  final  thing  to  be 
decided  in  sampling  is  how  the  sample  will  be  drawn.  The 
objective  in  the  method  is  to  insure  that  the  sample  is 
random  (that  is,  that  all  services  have  an  equal  chance  of 
being  selected).  To  achieve  random  selection,  use  a  random 
number  table,  as  explained  in  the  following  examples  (see 
attachment  1  for  the  whole  table) .  Most  items  will  fall 
into  one  of  these  examples. 
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(3)  Go  to  the  next  number,  60756.  The  last  part  of 
this  number,  756,  falls  within  the  brackets  one  is  looking 
for,  so  workorder  756  is  selected  to  be  sampled.  The  next 
random  number  group  is  92144.  Since  144  is  not  within  the 
brackets,  move  to  the  next  group  49442.  Again,  442  is  not 
within  the  brackets  and  therefore  is  not  selected  to  be 
sampled . 

(4)  This  process  would  be  continued  until  three 
workorders  are  selected. 

c.  How  To  Use  The  Random  Number  Table  To  Identify 
Random  Sample  From  a  List.  If  a  number  of  items  need  to  be 
sampled  that  are  not  consecutively  numbered,  the  simplest 
solution  is  to  list  the  identifiers,  for  all  the  items  in  the 
lot,  in  a  column,  on  a  piece  of  lined  paper. 

(1)  Next,  number  the  lines  consecutively,  beginning 
with  the  number  one  hundred.  Now  use  the  random  number  table 
to  draw  the  sample  from  the  line  numbers.  A  selected  line 
number  leads  to  the  identifier  located  on  that  line  and  that 
identifier  tells  which  item  to  sample.  For  example,  if  one 
chooses  to  sample  a  set  of  workorders  with  attached  sales 
slips,  one  is  not  going  to  have  to  have  a  set  of 
consecutively  numbered  workorders  because  not  every  workorder 
has  a  sales  slip  attached. 

(2)  List  the  workorders  with  sales  slips  in  a 
column,  number  each  line  in  the  column,  and  randomly  select 
enough  line  numbers  to  make  up  the  sample. 

d.  How  To  Use  The  Random  Number  Table  To  Identify  a 
Random  Sample  of  Days.  Suppose  one  wants  to  identify  4  days 
in  the  month  on  which  to  sample  something.  The  days  of  the 
month  can  be  numbered  01  to  31  (or  less,  as  appropriate). 
Begin  in  the  random  number  table  in  figure  4-5  at  77452. 

(1)  It  is  best  to  use  a  starting  point  different 
from  the  one  used  in  the  previous  example  but  for  the  purpose 
of  this  example  it  is  being  used  again. 

(2)  One  can  move  down  the  column  from  random  number 
group  to  random  number  until  the  first  number  between  01  and 
31  is  spotted.  In  this  case,  it  is  23216  or,  using  the  rule 
to  discard  the  numbers  to  the  left  of  the  number  of  digits, 
simply  16.  Thus  the  16th  day  of  the  month  is  selected  for 
sampling . 

(3)  Continuing  in  this  fashion,  one  discovers  that 
58731,  or  simply  31,  or  the  31st,  is  the  next  day  for 
sampling.  Proceed  in  this  manner  until  the  four  days  for 
sampling  have  been  identified. 
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(4)  If  it  is  not  desirable  to  sample  on  weekends, 
discard  those  days  selected  that  happen  to  fall  on  a  weekend 
and  continue  that  selection  until  the  proper  number  of  days 
has  been  selected. 

e.  How  To  Use  the  Random  Number  Table  To  Identify  a 
Random  Sample  of  Times  of  Day.  If  one  wants  to  select  random 
times  of  day  to  sample  a  service  such  as  taxi  or  bus  service, 
use  the  24  hour  clock. 

(1)  If  there  are  any  constraints  during  each 
24-hour  period,  take  them  into  consideration.  For  example, 
suppose  that  base  bus  service  operates  between  0700  and  2345. 
In  this  case,  go  through  the  number  table  until  one  finds  a 
group  of  four  numbers  that  correspond  to  an  acceptable  time 
between  0700  and  2345.  Again,  using  figure  4-5,  and 
proceeding  across  the  line  from  the  initial  number,  one  comes 
to  60756  or  0756  hrs  as  the  first  random  time. 

(2)  The  next  random  number  is  92144  or  2144  hrs. 

The  number  is  good  and  so  one  schedules  an  observation  for 
2144  hrs. 


(3)  Proceed  in  this  manner  until  the  desired  number 
of  sample  times  have  been  identified. 

f.  How  To  Insure  Variety  in  the  Use  of  the  Random 
Number  Table.  The  use  of  variety  in  the  random  number  table 
ensures  that  detectable  patterns  do  not  occur. 

(1)  Besides  starting  at  different  random  points  and 
alternating  the  patterns  for  finding  a  string  of  random 
numbers,  the  user  may,  at  some  point  in  time,  wish  to  use  the 
first  significant  digits  instead  of  the  last, 

(2)  For  instance,  in  the  random  number  group  77452 
one  has  customarily  used  the  last  three  digits  (that  is,  452) 
when  looking  for  a  random  number  with  three  digits.  But 
there  is  no  reason  why  one  could  not  for  a  period  of  time  use 
the  first  three  digits,  or  774. 

(3)  Success  in  using  the  tables  requires 
consistency  but  also  variety.  The  above  information  should 
ensure  that  the  tables  are  properly  used  and  that  the  sample 
is  randomly  drawn. 

4-5.  Surveillance  Plan  Products.  Several  written  documents 
are  included  in  a  surveillance  plan: 

a.  Sampling  Guides.  A  sampling  guide  is  used  for 
surveillance.  It  is  used  in  a  surveillance  plan  to  present 
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a.  Use  Of  The  Random  Number  Table.  The  random  numbers 
in  figure  4-5  are  arranged  in  groups  of  five  numbers  (51259, 
77452,  and  so  on) . 

(1)  To  use  the  table,  begin  by  picking  at  random  a 
group  of  numbers  on  any  page  of  the  table.  This  is  usually 
done  by  closing  the  eyes  and  pointing  with  a  pencil  or 
finger  to  some  initial  group. 

(2)  To  identify  additional  random  numbers,  follow 
a  pattern.  Go  along  a  given  line  to  ics  end  and  then  along 
the  next  line  to  its  end  and  so  on  through  the  table  until 
enough  numbers  have  been  selected  or  until  the  table  ends. 

(3)  If  the  table  ends  and  there  are  still  more 
numbers  to  select,  go  back  to  the  beginning  of  the  table  and 
continue  using  the  same  pattern.  Use  various  patterns 
alternately;  for  example,  use  lines  for  one  sample,  use 
columns  for  the  next  sample,  and  use  a  diagonal  pattern  for 
the  third  sample. 

b.  How  To  Use  the  Random  Number  Table  To  Identify  a 
Random  Sample  of  Consecutively  Numbered  Workorders.  Suppose 
one  has  to  identify  a  random  sample  of  3  workorders  for 
inspection.  This  can  be  done  at  the  beginning  of  the  month 
(before  the  workorders  are  written)  or  at  the  end  of  the 
month  (to  select  workorders  already  on  file). 

(1)  If  there  are,  or  might  be,  200  workorders  to 
select  from,  then  one  begins  by  listing  the  lowest  workorder 
number  (known  or  projected).  This  could  be  #001  or  possibly 
#743,  for  example. 

(a)  List  the  highest  workorder  number  (known 
or  projected);  in  this  case,  it  could  be  #200  or  #943.  With 
these  boundaries  now  enter  the  random  number  table  to  the 
first  group  of  numbers.  For  this  example,  use  workorders 
numbered  #743  to  #943. 

(b)  If  the  last  three  digits  in  the  first 
group  of  random  numbers  is  not  between  743  and  943,  discard 
that  group  of  numbers  and  go  to  the  next  group. 

(2)  Again,  using  figure  4-5,  if  one  starts  at  the 
initial  77452,  disregard  the  two  numbers  to  the  left  of  the 
three  significant  digits,  or  in  this  case,  77.  The 
remaining  number  is  452.  Since  this  is  not  between  743  and 
943,  go  to  the  next  group  in  the  same  line  which  is  16308, 
again,  discard  the  leftmost  two  numbers,  and  the  number  is 
308.  This  is  again  too  low. 
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Figure  4-5.  How  To  Use  A  Random  Number  Table. 


the  information  needed  to  sample  the  performance  of  a 
particular  service.  Information  for  the  sampling  guide  is 
developed  while  the  sampling  information  is  being  derived  for 
the  sampling  plan.  The  steps  involved  in  developing 
sampling  information  are  described  in  paragraph  4-3.  As 
shown  in  figure  4-6,  a  sampling  guide  has  these  sections: 

(1)  A  statement  of  the  AQL  ">nd  its  meaning  in 
layman's  terms. 

(2)  The  lot  size  for  sampling. 

(3)  The  sample  size. 

(4)  A  description  of  the  sampling  procedure 
which  tells  how  the  service  will  be  sampled. 

(5)  An  explanation  of  the  inspection  procedure 
which  tells  what  will  be  checked  during  the  inspection  of  the 
sample. 

(6)  Acceptable  performance  criteria  which 
states  the  acceptance  and  rejection  levels. 

b.  QAE  Decision  Tables.  Once  a  problem  has  been 
discovered,  the  inspector  must  turn  to  a  decision  table  and 
use  the  information  in  that  table  to  aid  him  in  finding  the 
source  of  the  problem.  The  decision  table  lists  the  symptoms 
of  the  problem  and  identifies  the  possible  sources  of  the 
problem.  Questions  are  established  for  each  potential  source 
to  determine  the  contributing  factors.  A  decision  logic 
entry  is  worked  up  for  each  required  service.  As  soon  as  it 
is  considered  satisfactory,  the  information  is  transferred  to 
the  decision  table.  An  example  of  a  decision  logic  entry  is 
shown  as  part  of  a  decision  table  in  the  sample  in  figure 
4-7. 

c.  Checklists.  There  are  two  main  uses  for  checklists. 

(1)  Tally  Checklists.  Tally  checklists  are  used  to 
document  all  sample  observations  made  during  a  sampling 
period.  Checklists  may  be  preprinted  with  any  format  which 
contains  the  following  information: 

(a)  Contract  requirements  -  a  statement  of  the 
service  being  inspected. 

(b)  Date,  time,  entry  for  each  observation. 

(c)  Observation  identifier  of  applicable 
workorder  number,  bus  stop,  or  sales  slip  number,  meal 
period,  etc. 


VO  Sampling  Guide  #3 
Vehicle  Condition  Monitoring 

1.  Acceptable  Quality  Level  (AQL) :  102 

In  the  long  run  there  must  be  no  more  than  10  defects  per- hundred 
vehicles. 

2.  Lot  Size:  _ vehicles  operated  by  the  contractor. 

3.  Sample  Size:  _ vehicles  operated  by  the  contractor. 

4.  Sampling  Procedure: 

At  the  beginning  of  the  month,  list  the  registration  numbers  of  all 
contractor  operated  vehicles  on  a  sheet  of  ruled  paper.  Beginning  with 
the  number  100,  number  the  lines  on  the  paper  to  correspond  with  the 
vehicle  registration  numbers.  Using  the  random  number  table  select  line 
numbers  equal  to  the  sample  size.  The  vehicle  registration  numbers  on 
these  lines  indicate  the  vehicles  to  be  sampled  during  the  month.  Schedule 
the  inspections  evenly  over  the  month. 

5.  Inspection  Procedure: 

Inspect  the  vehicles  using  vehicle/ equipment  discrepancy  and 
maintenance  report  as  a  guide  (see  AFM  77-310,  Vol  II,  Chap  6).  Record 
defects  per  vehicle  for  each  of  the  inspected  vehicles.  Any  defects 
found  not  already  noted  by  the  contractor  shall  cause  the  observation 
to  be  recorded  as  unsatisfactory. 

6.  Performance  Criteria: 

a.  Performance  is  acceptable  when  _  or  less  defective  vehicles  are 

discovered  per  month. 

b.  Performance  is  unacceptable  when  _ or  more  defective  vehicles 

are  discovered  during  a  month.. 

7.  Phase-In  Period:  During  the  first  two  months  of  the  contract  the 
following  AQL  s  (paragraph  1)  and  performance  criteria  (paragraph  6) 

“KP 'J • 

a.  AQL:  152 

b.  Performance  is  acceptable  if  _ or  fewer  defects  are  discovered 

per  month. 

c.  Performance  is  unacceptable  if  _  or  more  defects  are 

discovered  per  month. 
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Figure  4-6.  Sampling  Guide. 
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Figure  4-7.  QAE  Decision  Fable. 


(d)  Result  of  observation  -  either 
satisfactory  or  defective. 

(e)  Any  pertinent  comment  for  an  observation. 
An  example  of  a  checklist  for  sampling  is  shown  in  figure 

4-8. 


(2)  Surveillance  Activity  Checklists.  When  a 
specific  service  or  procedure  occurs  rarely  or  is  not 
important  enough  to  survey  on  a  continuing  basis,  use  a 
Surveillance  Activity  Checklist.  This  type  of  checklist  must 
be  prepared  and  included  in  the  surveillance  plan.  An 
example  of  this  kind  of  checklist  is  shown  in  figure  4-9. 

4-6.  Contract  Administrator's  Plan.  The  contract 
administrator  has  an  obligation  to  see  how  well  the  QAE  is 
doing  the  job.  The  contract  administrator  must  also  make 
some  independent  checks  of  contractor  performance,  preferably 
by  using  the  same  techniques  that  go  into  the  design  of  the 
QA  surveillance  plan.  (This  plan  is  completed  at  the  same 
time  as  the  surveillance  plan.)  As  a  minimum  this  plan  must 
call  for  a  quarterly  review  of  the  QAE's  use  of  sampling 
guides  and  an  annual  review  of  surveillance  activity 
checklist  items. 


Vehicle  Operations  Sampling  Guide  #3 


Vehicle  Condition  Monitoring 


Registration 

Number 


Date 


Time 


Satisfactory  Un_ 

satisfactor; 


Remarks 


B  7305 


2  Oct 


1530 


3  9763 


2  Oct 


1540 


Hood  latch  KLG 


B  8764 


2  Oct 


1545 


B  0010 


5  Oct 


0900 


B  8764 


5  Oct 


0915 


8  7707 


5  Oct  I  1345 


B  7706 


5  Oct 


1400 


B  9654 


8  Oct 


1000 


B  8752 


B  3103 


B  2753 


8  Oct 


1025 


8  Oct 


1045 


8  Oct 


1600 


Door  handle  KLG 


Figure  4-8.  Sample  Tally  Checklist. 
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CHAPTER  5 


DOING  SURVEILLANCE 


5-1.  Surveillance  Methods.  This  chapter  tells  how  to  do 
surveillance  once  the  plan  is  written.  It  tells  how  to  build 
a  monthly  schedule,  how  to  use  the  surveillance  plan,  and 
what  to  do  when  there  is  poor  contractor  performance.  This 
chapter  applies  to  Quality  Assurance  Evaluators  (QAEs)  and 
contract  administrators. 

5-2.  Building  A  Schedule.  A  surveillance  plan  is  organized 
to  facilitate  use  by  the  QAE.  The  QAE  is  responsible  for 
developing  a  monthly  schedule  for  activities,  based  on  the 
surveillance  plan's  requirements.  Complete  the  Quality 
Assurance  Evaluator  Schedule  by  the  last  workday  of  the 
preceding  month  and  send  a  copy  to  the  contract  administrator 
and  the  functional  area  chief  for  their  information  and 
review.  Each  QAE  builds  a  schedule  by  filling  in  the  blocks 
on  the  schedule.  Specific  instructions  for  filling  out  the 
schedule  are  provided  below. 

a.  Quality  Assurance  Evaluator  Schedule.  An  example  of 
a  surveillance  schedule  is  show  in  figure  5-1.  The  left-hand 
side  of  the  schedule  divides  the  sheet  into  days  of  the  week. 
This  example  shows  only  a  7-day  schedule.  The  QAE  must  make 
up  enough  sheets  to  include  each  day  of  the  month.  Along  the 
top  of  the  schedule,  insert  the  items  to  be  checked  during 
the  month.  Along  the  bottom  of  the  schedule,  indicate  the 
number  of  observations  to  be  made  during  the  month  (that  is, 
how  often  a  MIS  is  checked,  how  many  samples  will  be  taken, 
how  often  a  surveillance  checklist  will  be  used). 

b.  Filling  In  And  Updating  the  Schedule.  To  fill  in 
the  inspection  schedule,  the  QAE  refers  to  the  sampling  guide 
for  each  service  being  monitored.  The  sampling  guide  is  used 
with  the  random  number  table  to  determine  the  inspections 
(observations)  to  be  made  during  the  month  (see  chapter  4, 
paragraph  4-4). 

(1)  Contract  surveillance  must  cover  all  hours  of 
operation.  Random  observations  are  scheduled  at  night,  on 
weekends  and  holidays  when  services  are  performed  during 
these  periods.  Areas  that  are  monitored  on  a  set  schedule 
(for  example,  VIMS  standards  and  analysis  reports)  are 
included  in  the  monthly  schedule.  This  monthly  schedule 
shows  where  and  what  the  QAE  is  monitoring  at  all  times. 


(2)  Post  any  changes  to  the  schedule  weekly  and 
send  copies  to  the  contract  administrator  and  to  the 
functional  area  chief.  Document  and  explain  the  reasons  for 
each  change.  Actual  surveillance  activity  recorded  on  the 
surveillance  checklist  must  be  comparable  to  the  monthly 
schedule. 

(3)  As  updated,  one  must  be  able  to  conduct  a 
complete  audit  trail  from  the  monthly  schedule,  to  observing 
the  QAE  perform  sampling,  to  completion  of  the  surveillance 
checklist. 

(a)  There  must  also  be  a  correlation  between 
contractor  performance  versus  standards,  AQLs ,  checklists  and 
actions  taken  by  the  contract  administrator.  The  sample  in 
figure  5-1  shows  the  schedule  for  one  week.  The  QAE 
completes  the  blank  forms,  indicating  week  of  (Monday  through 
Sunday),  and  enters  the  time,  observation,  and  check  (if 
pertaining  to  a  checklist) ,  in  the  blocks  corresponding  to 
the  item  and  day. 

(b)  After  it  is  completed  and  filled  in,  this 

form  is  "FOR  OFFICIAL  USE  ONLY"  and  must  not  be  shown  to  the 
contractor.  ~  " 

5-3.  Doing  Surveillance.  Doing  surveillance  involves  using 
the  surveillance  plan  called  for  in  the  monthly  schedule. 

Use  the  following  procedure  to  record  observations  and  take 
action  when  the  contractor's  defects  exceed  the  allowable 
number. 

a.  Recording  Observations.  Monthly  tally  and 
surveillance  checklists  are  used  for  each  sampling  guide 
and  less  frequently  checked  services.  They  are  used  to  tally 
information  on  scheduled  observations  and  defects  noted. 

Each  observation  in  the  sample  is  recorded  on  the  checklists, 
and  the  documents  then  become  a  formal  government  record  for 
later  reference. 

(1)  When  random  sampling  guides  are  used,  the  tally 
of  observations  and  defects  at  the  end  of  each  month  are 
compared  to  the  acceptable  number  of  defects  appearing  in  the 
sampling  guide. 

(2)  The  contractor  is  told  each  time  an  error  is 
found  during  scheduled  observations  and  asked  to  initial  the 
observation  recorded  on  the  checklist. 

(3)  Errors  found  in  services  not  scheduled  for 
observation  should  be  brought  to  the  contractor's  attention 
but  not  used  to  count  as  a  defect  for  determining  if  the  AQL 
has  been  met. 


(4)  Checks  done  with  a  surveillance  activity 
checklist  are  likewise  recorded. 

b.  Potential  Unsatisfactory  Performance .  If  the 
sampling  guide  or  surveillance  activity  checklist  indicates 
that  the  number  of  defects  is  too  high,  the  QAE  goes  to  the 
decision  table  for  that  service  indicator. 

(1)  The  QAE  must  locate  the  specific  service  that 
is  unsatisfactory.  The  table  will  identify  the  possible 
causes  of  the  unsatisfactory  performance  and  list  a  number  of 
questions  which,  if  answered,  will  probably  pinpoint  the 
source  of  the  problem. 

(2)  The  decision  table  helps  the  QAE  identify  the 
problem  so  that,  among  other  things,  a  meaningful  evaluation 
can  be  made  of  the  contractor's  explanation  and  corrective 
action.  For  example,  if  the  contract  specifies  a  maximum 
out-of-commission  rate  for  vehicles  of  8  percent,  and  the 
rate  was  10  percent,  examination  may  reveal  the  excess  was 
caused  by  excessive  vehicle  down  for  parts  (VDP) .  This  could 
have  been  caused  by  the  government's  inability  to  provide 
timely  parts  support. 

(3)  In  such  a  situation,  the  contractor  may  not  be 
at  fault.  If,  on  the  other  hand,  the  excessive  VDP  was 
created  because  the  contractor  ordered  the  parts  on  a  routine 
priority  rather  than  priority,  it  might  be  the  contractor's 
fault.  The  decision  tables  will  assist  the  QAE  in  making 
such  a  determination. 

c.  Documenting  Unsatisfactory  Performance.  If 
performance  in  any  area  is  judged  unsatisfactory,  the 
contractor  is  required  to  respond  to  a  Contract  Discrepancy 
Report  (CDR).  See  sample  in  Figure  5-2. 

(1)  The  QAE  prepares  the  form  and  sends  it  to  the 
contracting  officer,  who  signs  and  sends  it  to  the 
contractor. 

(2)  When  completed  and  signed,  the  report,  along 
with  the  tally  checklist  or  surveillance  activity  checklist 
become  the  documentation  supporting  payment,  nonpayment,  or 
other  necessary  action. 

5-4.  Taking  Action.  The  QAE  may  check  the  contractor's 
performance  and  document  any  non-compliance,  but  only  the 
contracting  officer  may  take  formal  action  against  the 
contractor  for  unsatisfactory  performance. 


contract  discrepancy  report 


«a«  f  «»c  V  mw»m 

FT234S-79-9976S 


Reference  the  performance  requirements  summary  Exhibit  12.  The  contract  requires 
that  taxi  service  meet  a  four  minute  response  time  with  an  acceptable  quality  level 
of  151.  RanCom  observations  Indicate  that  this  standard  was  not  met.  In  a  sample 
of  20  dispatches,  to  pickups  exceeded  the  standard.. 


"*•••  Contracting  OSSiser 


Contracting  OCiicar) 


tallT  I#  KICtllaRY.  fCWt  C«*«  *f|W<d  9 


>  w  pe«  9-A. 


During  August,  there  was  a  limited  number  of  drivers  and  vehicles  available  due  to 
sickness  and  maintenance.  I  will  initiate  short  morning  coordination  meetings  each 
day  at  7:30  a.m.  so  that  maintenance  and  operations  personnel  can  assure  that 
enough  drivers  and  vehicles  are  available  for  daily  activities. 


Ip  MatMMIMT  lY*k^  O-w  <*«••*■ 

The  proposed  corrective  action  and  explanations  are  acceptable. 


»  |7«r»ww  Mteilw,  «ar*  Meat,  •**•*•! 


The  contractor's  actions  should  prevent  further  recurrence.  A  deduction  of  56750  will 
be  made  from  the  August  Invoice  computed  as  follows:  .lonthly  Cost  -  590, COC 

Deduct  Percentage  151 
Percent  of  Samde  Defective 


eduction  So/ 

If  this  recurs  next  month  a  deduction  will  be  taken  plus  a  cure  notice  will  be  Issued. 


a.  Ground  rules.  This  section  lists  the  normal  steps  to 
be  taken  by  contract  administration  when  the  QAE  reports 
these  deficiencies.  The  actions  listed  are  not  hard-and-fast 
rules,  and  are  a  minimum.  More  serious  action  can  be  taken 
sooner. 


(1)  When  the  contractor's  performance  is 
unsatisfactory  as  defined  in  the  surveillance  plan  and  a 
formal  action  is  indicated,  the  QAE,  the  functional  area 
chief,  and  the  contract  administrator  meet  to  determine  what 
action  is  appropriate  for  the  specific  circumstances. 

(2)  If  a  decision  is  reached  not  to  take  a  monetary 
deduction,  the  reasons  are  documented.  The  contracting 
officer  must  indicate  agreement  with  the  decision  by  signing 
the  contract  discrepancy  report  or  other  decision 
documentation. 

b.  Actions.  Following  are  the  actions  normally  taken 
when  poor  performance  is  found. 

(1)  As  a  rule,  the  QAE  tells  the  contractor's  site 
manager,  in  person,  when  discrepancies  occur  and  asks  the 
contractor  to  correct  the  problem.  The  QAE  makes  a  notation 
on  the  tally  or  surveillance  checklist,  of  the  date  and  time 
the  deficiency  was  discovered,  and  has  a  contractor 
representative  initial  the  entry  on  the  checklist. 

(2)  If  the  number  of  discrepancies  found  exceeds 
the  level  for  satisfactory  performance,  the  QAE  uses  the 
decision  tables  in  the  surveillance  plan  to  determine  the 
cause! s) . 

(a)  If  the  government  created  any  of  the 
discrepancies,  these  are  not  to  be  counted  against  the 
contractor's  performance. 

{ b)  When  the  government  has  caused  the 
contractor  to  perform  in  an  unsatisfactory  manner,  the  QAE 
prepares  a  letter  to  be  sent  to  the  responsible  organization 
requesting  corrective  action  be  taken.  The  QAE  sends  it  to 
the  organization  through  the  contracting  officer. 

(3)  When  the  contractor  is  responsible  for  exceeding 
the  limits  of  satisfactory  performance,  the  contracting 
officer  issues  a  contract  discrepancy  report  (CDR)  to  the 
contractor  (see  paragraph  5-3c)  .  If  the  failure  is  serious 
enough,  issue  the  CDR  at  the  time  of  the  unsatisfactory 
performance,  rather  than  at  the  end  of  the  month. 


(4)  When  a  CDR  is  issued  for  a  specific  service 
the  contracting  office  deducts  from  the  month's  payment,  an 
amount  up  to  the  percentage  indicated  in  the  Performance 
Requirement  Summary  exhibit  of  the  contract.  Do  not  delay 
the  deduction  until  the  contractor  responds  to  the  CDR.  If 
surveillance  was  done  right  and  the  decision  tables  used,  the 
unsatisfactory  performance  is  clearly  the  fault  of  the 
contractor.  For  a  specific  example  of  a  deduction,  see 
paragraph  5-5. 

(5)  If  the  contractor  does  not  achieve  satisfactory 
performance  in  that  specific  service  by  the  end  of  the  next 
month,  the  contracting  officer  issues  another  CDR  and  deducts 
the  appropriate  amount  from  the  contractor's  payment. 

(6)  If  a  third  CDR  must  be  issued,  consider  issuing 
a  cure  notice.  (However,  a  cure  notice  can  be  issued  sooner, 
if  necessary) . 

(7)  Depending  on  the  contractor's  overall 
performance,  the  government  may  issue  a  Show  Cause  letter  if 
the  reply  to  a  cure  notice  is  unsatisfactory;  next  consider 
terminating  the  contract. 

5-5.  Deductions  For  Non-Performance.  Through  the  Inspection 
of  Services  clause,  the  government  can  deduct  from  a 
contractor's  payment  an  amount  equal  to  the  services  not 
provided. 

a.  To  do  this,  the  contract  administrator  must  know  the 
major  cost  categories  in  the  contract  and  the  percentage  of 
cost  each  service  output  represents.  The  percentage  cost  of 
each  service  is  found  in  deduct  analysis;  see  chapter  2, 
paragraph  2-9.  An  example  of  how  the  deduct  formula  works  is 
shown  in  figure  5-3. 

b.  Suppose  the  bid  schedule  showed  the  monthly  contract 
price  for  vehicle  operations,  maintenance,  and  analysis  as 
shown.  The  percentage  cost  of  the  service  output  is  then 
found  by  looking  at  the  Performance  Requirements  Summary 
Technical  Exhibit  in  the  contract  statement  of  work.  In  the 
example,  the  percentage  cost  of  quality  of  completed  work  is 
10  percent.  This  is  then  multiplied  by  $100,000  to  obtain 
the  maximum  amount  to  deduct. 

c.  If  completed  work  was  unsatisfactory  during  the 
month  (that  is,  did  not  meet  performance  values)  and  the 
percent  of  the  sample  found  bad  was  20  percent,  $2000  would 
be  deducted  from  the  payment  normally  due  the  contractor. 
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Deduct  Formula  (Example) 


If:  Quality  of  completed  work  is  unsatisfactory 

(AQL  of  6.5%  exceeded) 

and:  Contract  price  is  5100,000  per  month 

and:  Quality  of  completed  work  deduct  percentage  is  10% 

and:  Sample  size  is  50 

and:  Number  of  defects  in  the  sample  is  10  (Reject  number  is  8) 

Then:  Deduction  from  the  current  month's  invoice  is: 


Contract  price 
X  Oeduct  percentage 
X  Percent  of  sample  defective 
Deduction 


$100,000 

.10 

$10,000 

.20 

$  2,000 


Figure  5-3.  Deducting  for  Non-Performance. 


d.  This  amount  for  quality  of  completed  work  is  deducted 
because  the  contractor  failed  to  provide  reliable,  uniform 
services  within  the  assigned  performance  values.  Although 
some  completed  work  may  have  met  the  standard  during  the 
month,  the  acceptable  quality  level  was  not  met  and  at  least 
20  percent  of  the  observations  were  defective.  Hence,  the 
total  quality  performance  requirement  has  not  been  achieved. 
As  a  consequence,  the  service  output  is  unsatisfactory. 

5-6.  Good  Performance.  When  a  contractor's  quality  control 
program  works,  good  performance  results.  If  the  result  of  a 
QAE 1 s  surveillance  shows  consistently  good  performance,  the 
amount  of  surveillance  can  be  decreased. 

a.  Reduced  Inspection.  Inspection  can  be  reduced 
when  the  following  conditions  have  been  met  for  a  sampling 
guide. 

(1)  The  preceding  4  lots  (that  is,  the  last  4 
months)  have  all  been  acceptable. 

(2)  The  number  of  defects  in  each  of  the  preceding 
4  lots  is  less  than  one  half  of  the  acceptance  number.  For 
example,  with  an  AQL  of  6.5  percent  and  a  sample  size  of  32, 
the  acceptance  number  is  5.  If  two  or  less  defects  were 
found  in  each  of  the  last  4  lots,  reduced  inspection  could  be 
used . 

(3)  The  normal  sample  size  is  being  used. 

(4)  The  functional  area  chief  and  the  contract 
administrator  agree  to  use  reduced  inspection. 

b.  Reduced  Sample  Size  and  Acceptance  or  Rejection 
Numbers.  Reduced  inspection  decreases  the  sample  size  as 
shown  in  figure  5-4.  In  addition,  the  acceptance  and 
rejection  numbers  change  as  shown  in  figure  5-5.  To  make  the 
changes  to  the  existing  sampling  guide,  take  the  following 

s  teps . 

(1)  Make  sure  that  the  original  sampling  guj.3e  was 
using  the  normal  sample  size.  To  determine  this,  see  Chapter 
4,  figure  4-3  and  compare  the  lot  size  with  the  sample  size 
in  the  sampling  guide. 

(2)  Find  the  new  sample  size  by  using  figure  5-4. 
Take  the  lot  size  and  find  the  new  reduced  sample  size. 

(3)  Using  the  AQL  in  the  sampling  guide  and  the  new 
reduced  sample  size,  see  figure  5-5  for  the  new  acceptance 


Lot  Size 


Normal  Sample  Size 


Reduced  Sample  Size 


I 


2-8 

2 

2 

9-15 

3 

2 

16-25 

5 

2 

26-50 

8 

3 

51-90 

13 

5 

91-150 

20 

8 

151-280 

32  . 

13 

281-500 

50 

20 

501-1,200 

80 

32 

1.201-3,200 

125 

50 

3.201-10,000 

200 

80 

10,001-35,000 

315 

125 

35,001-150,000 

500 

200 

150,001-500,000 

800 

315 

500,001  and  over 

1250  • 

500 

Figure  5-4.  Reduced  Sample  Size. 


Sitgle  sampling  plant  for  reduced  inspection 


Figure  5-5.  HIL-ST0-1050  Acceptance,  Rejection  Levels  for  Reduced  Inspection 


and  rejection  numbers.  Note  that  there  is  a  gap  between  the 
acceptance  and  rejection  numbers  (for  example,  sample  size  32 
and  AQL  6.5  percent,  accept  is  5  and  reject  is  8).  This  means 
that  the  lot  would  not  be  rejected  unless  8  defects  were 
found  and  would  be  accepted  if  5  or  less  defects  were  found. 
However,  a  number  of  defects  greater  than  five  will  be  cause 
for  returning  to  normal  inspection  (that  is,  return  to  the 
sample  size  and  acceptance  and  rejection  numbers  used  in  the 
original  sampling  guide)  . 

c.  Returning  to  Normal  Inspection.  When  reduced 
inspection  is  in  effect  return  to  normal  inspection  the  next 
month  under  the  following  conditions. 

(1)  When  the  number  of  defects  exceeds  the 
acceptance  number  under  reduced  sampling  or, 

(2)  The  functional  area  chief  and  the  contract 
administrator  deem  it  necessary  to  return  to  normal 
inspection. 

d.  Returning  to  Reduced  Inspection.  If  during  the 
first  month  of  the  return  to  normal  inspection,  the  number  of 
defects  found  is  again  less  than  50  percent  of  the  reject 
level,  a  return  to  reduced  inspection  may  be  done  the  next 
month.  If  the  number  of  defects  found  is  over  50  percent, 
then  normal  sampling  must  be  accomplished  until  4  months  of 
less  than  50  percent  of  reject  level  defects  are  found. 

5-7.  Documentation.  During  the  course  of  the  contract  the 
QAE  retains  a  copy  of  all  inspection  schedules,  tally 
checklists,  and  surveillance  activity  checklists.  At  the  end 
of  the  contract  period,  the  QAE  forwards  these  records  for 
inclusion  in  the  contract  file.  However,  when  a  specific 
service  becomes  unsatisfactory  during  a  surveillance  period, 
the  inspection  documentation  supporting  the  contract 
discrepancy  report  is  forwarded  to  the  contracting  officer  no 
later  than  5  working  days  after  the  end  of  the  previous 
performance  period. 


APPENDIX  D 

EXCERETS  FROM  NAVFAC  HO-327 


This  appendix  includes  various  portions  of  NAVFAC  MC-327 
entitled  "Service  Contracts  :  Specifications  and 

Surveillar.ee."  The  first  illustration,  the  Performance 
Requirements  Summary  (PRS)  ,  is  a  hay  and  comprehensive 
surveillance  document.  It  lists  each  item  of  work  to  be 
performed  under  the  contract,  its  required  standard  of 
performance,  the  method  of  inspection  for  the  item,  and  the 
acceptable  quality  level  (or  allowable  deviation)  . 

The  next  series  of  illustrations  is  Chapter  5,  Quality 
Assurance  Methods.  ~ach  style  of  inspection,  be  it  planned, 
random,  etc.,  is  explained.  Criteria  for  chocsing  an  appro¬ 
priate  method  is  provided.  Sample  service  inventory  work¬ 
sheets  and  surveillar.ee  schedules  are  presented. 

The  next  part  of  this  appendix  is  Chapter  6, 
Surveillance.  Administrative  responsibilities  and  procedures 
are  listed. 

Appendix  C  is  an  abbreviated  sample  quality  assurance 
plan  for  industrial  solid  waste  collection.  Appendix  D  is  an 
instruction  for  the  utilization  of  random  number  tables. 
Finally,  current  NAVFAC  lot  and  sample  tables  are  provided 
although  these  will  te  modified  by  new  ONR  sampling  tables. 
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TASK 


STANDARD  & 
INDICATOR 


METHOD  OP 
SURVEILLANCE 


AQL 


1.  Residential  waste 
collection  (Sect. 
00005,  Clause  3.2) 


-collection  on  day 
specified 
-no  debris 
-cans  left  at  curb, 
off  street 


-planned  sampling 
(primary) 

-validated  complaints 
(secondary) 


3  units 
per  100 


2.  Scheduled  industrial  -collected  within 


waste  collection 
(Sect.  00005, 
Clause  3.3) 


4  hr  of  specified 
time 

-no  debri3 

-containers  properly 
repositioned 


-random  sampling 
(primary) 

-validated  complaints 
(secondary) 


10« 


4. 


Unscheduled 
industrial  waste 
collection  (Sect. 
00005,  Clause  3.4) 

Ash  collection 
(Sect.  00005, 
Clause  3.5) 


-collected  within 
12  hr  of  request 
for  pick-up 


-collected  within 
12  hr  of  request 
for  pick-up 


-validated  complaints 


-validated  complaints 


5  per 
month 


3  per 
month 


Quality  control 
program  (Sect. 
00004,  Clause  7) 


-QC  program 
established 
-activity  in  QC 
files 

-QC  identified 
problems  resolved 


-planned  sampling 


zero 


FIGURE  4-1 

Performance  Requirements  Summary  Table 
(Sample) 


CHAPTER  5; 


QUALITY  ASSURANCE  METHODS 


5.  GENERAL.  This  section  describes  quality  assurance  evaluation 

methods  used  to  monitor  .Contractor  performance  on  service  contracts.  Before 
proceeding,  the  purpose  and  goals  of  QA  should  be  discussed. 

a.  When  the  Government  purchases  goods  or  services,  there  must  be 
some  means  provided  to  attest  to  the  value  received  for  monies  spent.  To  do 
this,  the  Government  must  be  able  to  confirm  that  the  quantity  and  quality 
of  goods  or  services  received  conforms  to  contract  requirements. 

b.  The  recipients  of  the  contracted  goods  or  services,  in  this  case 
Naval  shore  activities,  are  responsible  for  developing  and  implementing 
procedures  that  assure  that  the  Government  is  getting  what  it  contracted 
for.  These  procedures  are  referred  to  as  QA. 

c.  Contractors,  on  the  other  hand,  are  responsible  for  providing 
Quality  Control  (QC) .  The  purpose  of  QC  is  to  control  the  service  producing 
process,  and  to  insure  that  the  desired  level  of  output  quality  is 
maintained. 

5.1  QA  PROGRAM. 

5.1.1  Traditional  Approach.  The  Navy's  traditional  approach  to 
surveillance  of  Service  Contracts,  often  a  hit-or-miss  affair  with  no 
written  plan,  has  not  provided  adequate  quality  assurance. 

a.  The  method  of  surveillance  which  is  claimed  to  be  used  most 
frequently  is  100  percent  inspection.  In  reality,  however,  the  inspection 
is  often  much  less  than  total,  since  100  hundred  percent  inspection  i3  very 
costly  and  not  always  feasible. 

b.  Further,  traditional  surveillance  methods  have  usually  focused 
on  the  work  process  (adherence  to  specified  3teps  and  frequencies)  rather 
than  on  the  quality  of  contract  outputs.  The  net  result  does  not  assure 
satisfactory  quality  performance. 

5.1.2  New  QA  Approach.  The  new  QA  approach,  based  on  a  written  plan,  i3 
keyed  to  performance  oriented  specifications. 

a.  It  focuses  on  the  quality  of  the  product  delivered  by  the 
Contractor  and  not  on  the  steps  taken  or  procedures  used  to  provide  that 
product. 

b.  It  includes  appropriate  use  of  preplanned  inspections, 
validation  of  complaints,  and  unscheduled  inspections. 

c.  It  provides  a  structured  approach  to  surveillance  that  permits 
management  control  of  QA. 

5.1.3  Criteria.  There  are  several  criteria  for  good  QA. 

a.  First,  the  PWS  must  be  written  so  that  the  quantity  and  quality 


of  required  work  outputs  are  measurable.  The  development  of  the  PWS  and  the 
QA  should  be  viewed  as  a  single  process.  These  documents  are  interrelated; 
one  defines  required  work  outputs  and  quality  standards  while  the  other 
defines  how  vrerk  outputs  will  be  observed  and  measured. 

b.  QA  must  provide  for  adequate  and  affordable  contract 
surveillance.  The  depth  and  detail  of  surveillance  should  be  geared  to  the 
relative  importance  of  the  services  monitored. 

c.  QA  must  have  the  potential  to  support  corrective  action  taken  by 
the  SCM/RQIC  when  non-performance  or  unsatisfactory  performance  occurs. 

d.  There  are  three  key  ideas  that  are  the  basis  for  contract 
surveillance. 

(1)  Outputs.  Quality  Assurance  evaluates  the  output  service 
provided  by  the  Contractor.  The  output  service  can  result  either  from  a 
Contractor  developed  procedure  or  from  a  Government  specified  procedure. 

When  the  output  is  based  on  a  Contractor  developed  procedure,  the  procedure 
is  examined  on  an  exceptions  basis;  that  is,  satisfactory  service  output  as 
specified  in  the  contract  normally  indicates  that  the  Contractor  is  using 
satisfactory  procedures.  The  Government  should  be  concerned  with  Contractor 
procedures  only  when  services  are  not  adequate.  When  the  procedure  is 
specified  by  the  Government,  compliance  with  the  procedure  is  the  desired 
output  service. 

(2)  Compliance.  Contractor's  compliance  with  contract 
requirements  is  monitored  through  the  performance  indicators  and  standards 
which  are  specified  in  the  PWS.  Performance  indicators  are  measurable 
attributes  of  the  outputs.  A  standard  is  the  gauge  that  Contractor 
performance  is  compared  against.  For  example,  scheduled  trash  collection  is 
the  work  required,  one  of  the  indicators  of  good  performance  is  timeliness, 
and  the  standard  is  trash  pick-up  is  made  within  4  hours  of  the  scheduled 
time. 

(3)  Cause  of  Problem.  When  observed  performance  indicators 
show  output  not  to  be  in  compliance  with  contract  requirements,  the  QAE 
identifies  the  cause  of  the  problem.  The  QAE  looks  beyond  service  outputs 
to  determine  if  the  problem  is  caused  by  the  Government  or  the  Contractor. 

If  the  cause  of  the  problem  rests  with  the  Government,  corrective  action 
must  be  taken  through  Government  channels  ar.d  r.o  action  is  required  uf  the 
Contractor.  If  the  Contractor  is  at  fault,  the  Contractor  is  told  to  take 
corrective  action;  payments  to  the  Contractor  are  reduced;  and  a  Contracts 
Discrepancy  Report  (CDR)  may  be  issued. 

5.2  METHODS  OP  SURVEILLANCE.  There  are  five  methods  that  may  be  used 
for  contract  surveillance. 

5.2.1  One  Hundred  Percent  Inspection.  One  hundred  percent  inspection 
requires  that  output  from  each  and  every  work  occurrence  be  evaluated.  One 
hundred  percent  inspection  measures  the  Contractor's  true  level  of 
performance.  It  is  an  expensive  and  time  consuming  method  which  should  be 
used  sparingly. 


5.2.2  Planned  Sampling.  Surveillance  by  planned  sampling  is  designed  to 
evaluate  a  part  but  not  all  of  a  contract  requirement. 

a.  The  number  of  inspections  and  the  items  to  be  inspected  are 
based  on  subjective  judgment. 

b.  Planned  sampling  is  useful  when  requirements  at  one  location  are 
more  important  than  those  at  other  locations;  for  example,  galley  dumpsters 
as  opposed  to  those  in  a  remote  administrative  area.  It  is  also  useful  when 
the  Contractor's  performance  is  poor  in  some  spot3  but  better  in  others. 

5.2.3  Random  Sampling.  Surveillance  based  on  random  sampling  evaluates 
part  but  not  all  of  the  work  performed.  (Appendix  0  provides  details  of  the 
mechanics  of  random  sampling.) 

a.  Using  random  sampling,  any  occurrence  of  work  i3  as  likely  to  be 
monitored  as  any  other  occurrence.  The  QAE's  bias  does  not  affect  the 
specific  occurrences  of  work  selected  for  evaluation  since  all  occurrences 
of  an  item  of  work  are  assumed  to  have  the  same  level  of  importance. 

b.  This  method  estimates  the  Contractor's  overall  level  of 
performance  for  a  given  contract  requirement.  It  is  mo3t  useful  when 
evaluating  items  that  are  repetitive  in  nature  such  as  janitorial  work, 
grounds  maintenance,  or  service  call  work. 

c.  To  achieve  full  benefits,  the  random  sampling  method  must  be 
applied  properly.  If  misapplied,  results  will  be  biased. 

5.2.4  Validated  Complaints.  Validated  customer  complaints  constitute  a 
surveillance  method  based  on  customer  awareness.  Customers,  familiar  with 
contract  requirements,  notify  the  QAE  when  there  is  a  case  of  poor  or  non¬ 
performance.  Upon  notification,  the  QAE  investigates  the  report  and,  if 
valid,  documents  the  performance  problem. 

a.  Formal  customer  complaints  are  a  means  of  documenting  certain 
kinds  of  service  problems.  The  way  to  obtain  and  document  customer 
complaints  needs  to  be  carefully  planned  by  the  persons  monitoring  the 
service  contract. 

b.  Customer  complaints  are  not  random,  but  when  validated  by  the 
QAE  may  be  used  to  deduct  money  from  the  Contractor. 

c.  When  random  sampling  is  the  chosen  method  of  surveillance,  a 
customer  complaint  cannot  be  used  to  satisfy  a  random  observance.  However, 
it  can  be  used  a3  further  evidence  of  unsatisfactory  performance  if  random 
sampling  shows  that  the  specific  service  is  unsatisfactory.  These 
complaints  can  be  used  to  decide  if  other  action  should  be  taken. 

(1)  Getting  Customer  Complaints.  An  aggressive  customer 
complaint  program,  once  established,  needs  to  be  explained  to  every 
organization  that  receives  the  Contractor '3  services.  An  operating 
instruction  should  be  given  to  each  organization  outlining  the  customer 
complaint  program,  the  format  and  the  content  of  a  formal  customer 
complaint,  and  the  action  which  can  be  expected  from  those  assigned  to 
watching  and  managing  the  service  contract.  (Appendix  E  provides  a 
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Customer's  Guide  for  Evaluating  Contractor  Performance.) 

(2)  Documenting  the  Customer  Complaint.  Normally,  each 
customer  complaint  is  brought,  in  person  or  by  telephone,  to  the  individual 
checking  contract  performance.  Enter  complaint  information  into  a  Customer 
Complaint  Record,  similar  to  the  sample  show  in  Figure  5-1.  The  record 
contains  the  following  information: 

.1  Date  and  time  of  complaint. 

2  Source  of  complaint  -  organization  or  individual. 

2  Nature  of  complaint  (narrative  description). 

£  Contract  reference  of  complaint  related  services. 

£  Valid  complaint  (Yes  or  No). 

6  Date  Contractor  informed  of  complaint. 

2  Action  taken  by  Contractor. 

S  Signature  of  the  person  receiving  and  validating  the 
complaint. 

5.2.5  Unscheduled  Inspections.  The  QAE  may  conduct  impromptu  evaluations 
of  contract  requirements  whenever  necessary.  This  surveillance  method 
(which  is  not  really  a  method  at  all)  provides  no  information  on  the 
Contractor's  overall  performance. 

5.2.6  Criteria  for  Method  Selection.  No  firm  guidance  for  method 
selection  can  be  provided  in  a  manual  of  this  type.  Some  general  guidance 
on  selection  criteria  are: 

a.  Population  size  refers  to  the  number  of  scheduled  or  expected 
occurrences  of  a  contract  requirement  over  a  given  time  period,  usually  one 
month.  The  actual  number  of  occurrences  will  depend  on  how  a  unit  of 
service  is  defined.  Frequency  of  service  at  any  location  may  be  daily, 
weekly,  etc.  Population  size  i3  easy  to  determine  for  scheduled  services. 
When  services  are  performed  on  an  "as  required"  basis,  population  size  must 
be  estimated  based  on  historical  or  projected  data.  Large  homogeneous 
populations  are  ideally  suited  to  random  sampling. 

b.  Some  contract  requirements  are  more  important  than  others.  Som 
requirements  may  have  an  impact  on  an  activity's  mission.  Others  have 
little  or  no  impact.  One  hundred  percent  inspection  might  be  used  for 
"important"  contract  requirements,  a  sampling  method  or  customer  complaints 
for  "less  important  requirements",  and  validated  complaints  for  the  "least 
important  requirements*. 

c.  It  is  easier  to  talk  about  instances  where  surveillance  methods 
are  not  appropriate. 


(1)  One-hundred  percent  inspection  is  not  suited  to  large 
populations.  It  would  be  time  consuming  for  the  QAE  and  expensive  to  the 
Government. 

(2)  Those  ,  services  that  are  important  or  costly  require  tight 
surveillance.  Validated  complaints  do  not  guarantee  that  all  instances  of 
nonperformance  or  poor  performance  will  be  reported  in  a  timely  manner. 

(3)  If  a  service  is  required  but  individual  occurrences  are  of 
small  importance  (for  example,  emptying  a  trash  can),  it  is  normally  not 
beneficial  to  invest  a  great  amount  in  surveillance.  One-hundred  percent 
inspection  is  the  most  costly  of  the  evaluation  methods  and  should  not  be 

used. 

(4)  If  a  contract  requirement  is  continuous  in  nature,  100 
percent  inspection  is  not  feasible  since  it  would  require  a  QAE  to  be 
on-site  full  time.  Examples  of  continuous  requirements  are  manning  a  guard 
post,  maintaining  a  minimum  inventory  of  parts,  and  fulfilling  contractor 
quality  control  requirements. 

C 5)  It  is  not  possible  to  use  a  surveillance  method  that 
requires  prescheduling  of  evaluations  for  unscheduled  service  3uch  as 
responding  to  Emergency/Service  (E/S)  calls,  processing  work  orders,  and 
dispatching  vehicles.  It  is  possible  to  schedule  retrospective  evaluation 
of  Management  Information  Systems  (MIS)  outputs  such  as  logs,  work  orders, 
or  other  written  records. 

d.  The  choice  between  planned  sampling  and  random  sampling  is 
sometimes  difficult. 

(1)  For  example,  on  a  Bus  Service  Contract  it  would  be  very 
easy  to  establish  a  random  sampling  evaluation  plan  where  the  QAE  monitors 
the  Contractor '3  compliance  with  the  established  bus  schedule.  However, 
this  type  of  surveillance  plan  would  require  the  QAE  to  travel  to  all  areas 
of  the  bus  activity  on  a  random  basis,  resulting  in  a  large  amount  of 
unproductive  QAE  travel  time.  Because  of  the  many  customer  complaints  that 
would  result  if  there  were  poor  bus  service,  planned  sampling  is  a  more 
attractive  surveillance  method. 

(2)  A  good  rule  of  thumb  in  choosing  between  random  and  planned 
sampling  is  that  if  all  evaluation  can  be  conducted  at  one  work  site,  random 
sampling  is  preferred.  If  work  sites  are  dispersed,  planned  sampling  should 
be  used.  On  the  other  hand,  increased  QAE  travel  time  may  be  a  good 
investment  if  the  work  item  is  important. 

e.  Ideally,  QAE's  should  be  staffed  to  the  level  required  to 
support  the  selected  QA  Program.  In  reality,  the  QA  progra  ised  must 
accommodate  the  availability  of  QAE's.  A  combination  of  QA  methods  should 
be  considered  to  get  the  best  QA  program  possible  with  a  given  number  of 
QAE' 3. 

5.2.7  Outputs  Subject  to  Surveillance.  The  QAE  must  determine  what  output 
to  inspect  to  determine  the  Contractor's  overall  performance.  In  many 
instances  a  contract  requirement  will  generate  several  outputs.  Evaluating 


DATE 


TIME 


SOURCE  OF  COMPLAINT: 
Organization  - 

Individual  - 

NATURE  OF  COMPLAINT: 


CONTRACT  REFERENCE: 

COMPLAINT  VALIDATED: 

Date  -  Time  - 

CONTRACTOR  INFORMED  OF  COMPLAINT: 

Date  -  Time  - 

ACTION  TAKEN  BY  CONTRACTOR: 


WORK  REINSPECTED: 

Date  -  Time  - 

Satisfactory  - 
Unsatisfactory  - 


FIGURE  5-1 

Customer  Complaint  Record 


a  single  output  may  provide  a  reliable  indication  of  the  Contractor's 
overall  performance.  Types  of  outputs  subject  to  inspection  are: 

a.  Work  Performed.  Inspection  of  all  work  performed  is  the  best 
way  to  evaluate  Contractor  performance.  This  requires  that  the  QAE  be  at 
the  work  site  during  performance  of  the  work,  or  be  there  shortly  thereafter 
if  results  of  work  performed  are  visible  (e.g. ,  janitorial,  grounds,  etc.). 

b.  Records.  If  the  QAE  is  unable  to  be  at  the  work  site  at 
required  times,  the  inspection  of  records,  work  chits,  and  other  documents, 
(combined  with  spot  checks  of  actual  work  performance,)  may  provide  a 
satisfactory  indication  of  work  performance. 

c.  Management  Information  System  (MIS) .  MISs  usually  collect 
information  over  a  specified  period  of  time.  This  information  can  be 
compared  to  a  contract  standard.  On  the  basis  of  this  comparison, 
performance  can  be  evaluated  and  the  performance  for  the  specified  period 
can  be  accepted  or  rejected. 

(1)  For  example,  the  vehicle  down  time  rate  is  computed  every 
month.  A  simple  comparison  of  the  rate  with  the  maximum  acceptable  rate  in 
the  SOV  explains  a  great  deal  about  the  level  of  maintenance  service 
supporting  the  base  vehicles. 

(2)  By  way  of  caution,  one  i3  going  to  U3e  a  MIS  to  check  the 
Contractor,  make  sure  the  MIS  contains  reliable  data. 

5.2.3  Inventory  of  Services.  Each  service  requirement  that  i3  to  be 
monitored  must  have  an  Inventory  of  Services  worksheet  prepared.  Figure  5-2 
is  a  sample  of  an  inventory  worksheet.  Inventory  worksheets  are  prepared  at 
the  same  time  the  QA  Plans  are  prepared. 

a.  Purpose.  Inventory  of  Services  Worksheets  serve  two  purposes. 
First  and  most  obvious,  these  worksheets  provide  a  comprehensive  listing  of 
locations  receiving  a  given  service.  Second,  these  worksheets  are  used  to 
select  locations  for  inspection  when  one  of  the  sampling  methods  i3  used. 

b.  Numbering.  Worksheets  list  each  location  where  a  service  will 
be  performed.  Work  occurences  should  be  grouped  as  daily,  weekly,  or 
monthly  depending  on  how  surveillance  is  to  be  conducted  u.e. ,  daily, 
weekly,  monthly,  or  other.)  Each  work  occurence,  within  the  group,  3hould 
then  be  sequentially  numbered. 

5.3  QA  PLANS.  The  most  important  part  of  implementating  a  QA  program  is 
the  development  of  comprehensive  QA  Plans.  (Appendix  C  provides  a  sample  of 
a  QA  Plan  for  Scheduled  Solid  Waste  Collection.) 

a.  QA  Plans  are  documentation  of  how  the  QAE  intends  to  monitoc 
specific  aspects  of  Contractor's  performance. 

b.  These  plans  are  the  basis  for  developing  QAE  schedules  and 
determining  required  QAE  staffing  levels. 


INVENTORY  OF  SERVICES  FOR:  Industrial/Administrative  Solid  Waste  Collection 
SURVEILLANCE  SCHEDULE:  Weekly 


LOCAT ION/1 DENTI F I CAT ION 


FIGURE  5-2 

Inventory  of  Services  Worksheet 
(Sample) 


c.  QA  Plans  are  developed  to  cover  all  items  shown  in  the 
Performance  Requirement  Summary  Table.  (A  single  plan  may  cover  more  than 
one  item  if  surveillance  of  those  items  is  compatible.) 

d.  QA  Plans  are  monitored  by  the  QAE  throughout  the  life  of  the 
contract.  Once  established,  they  do  not  tend  to  change. 

e.  Recommended  formats  for  QA  Plans  are  described  below.  (A  sample 
of  a  QA  Plan  is  provided  in  Appendix  C.) 

f.  Each  QA  Plan  established  should  have  an  evaluation  worksheet,  or 
checklist.  This  worksheet,  or  checklist,  is  used  to  document  surveillance 
results.  Figure  5-?  is  a  sample  of  an  Evaluation  Worksheet  for  refuse 
collection. 

5.3.1  100  Percent  Inspection. 

a.  Contract  Requirement.  Contract  requirements,  identified  in  the 
Performance  Requirements  Summary  Table  (see  Figure  4-1),  are  listed  along 
with  the  performance  indicators  and  standards.  Specific  clauses  in  the  SOW 
establishing  these  requirements  also  are  listed. 


SURVEILLANCE  SCHEDULE  FOR  INDUST RIAL/ ADMINS I STRATIVE  AREA  SOLID  WASTE  COLLECTION 
CONTRACT  NUMBER:  _ _ 


FOR:  22  June  1981  QAE:  I.  Peak 


BLDG/ 

LOCATION  Q4PTIED  DEBRIS  POSITION  CONDITION  COMMENT 


Bldg. 

#14 

p 

P 

P 

P 

Bldg. 

#19 

p 

F 

P 

P 

Trash  soilled  on  pavement 

when  dumpster  emptied 

Bldg. 

♦  28 

p 

P 

P 

P 

Bldg. 

#41 

F 

? 

P 

P 

Container  overflowing. 

Bldg. 

#43 

P 

P 

P 

P 

Area 

#  3 

P 

P 

P 

F 

Container  lid  will  not 

close 

Area 

#11 

P 

P 

P 

P 

Area 

#19 

P 

P 

P 

P 

Area 
♦  23 

P 

P 

P 

P 

:  : 

:  :  : 

t  ! 

FIGURE  5-3 
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b.  Primary  Method  of  Surveillance.  The  primary  method  of 
evaluation  is  100  percent  inspection. 

c.  AQL.  AQL's  are  best  stated  as  a  percentage. 

d.  Quantity  of  Work.  Define  a  unit  ( i .e . ,  single  occurrence)  of 
work,  and  determine  the  number  of  units  to  be  performed  during  the 
surveillance  period.  Performance  of  each  unit  will  be  evaluated. 

e.  Level  of  Evaluation.  This  is  not  applicable. 

f.  Sample  Size.  This  is  not  applicable. 

g.  Sample  Selection  Procedure.  This  is  not  applicable. 

h.  Evaluation  Procedure.  List  any  procedures  or  checklists  used 
when  doing  the  inspection.  Evaluation  procedures  should  be  in  enough  detail 
to  allow  others  to  continue  the  same  manner  of  inspection  using  the  same 
evaluation  criteria. 

i.  Analysis  of  Results.  The  analysis  of  evaluation  results  for  100 
percent  inspection  is  straight  forward.  The  Observed  Defect  Rate  (ODR)  is 
computed  as  follows: 


ODR  - 


Number  of  Defects  x 
Number  Units  of  Work 


5.3.2  Planned  Sampling. 

a.  Contract  Requirement.  Contract  requirements,  identified  in  the 
Performance  Requirements  Summary  Table  (see  Figure  4-1) ,  are  listed  along 
with  the  performance  indicators  and  standards.  Specific  clauses  in  the  SOW 
establishing  these  requirement  are  listed. 

b.  Primary  Method  of  Surveillance.  The  primary  method  of 
surveillance  is  planned  sampling.  Validated  complaints  are  a  good 
supportive  surveillance  method. 

c.  AQL.  AQL's  should  be  stated  as  an  absolute  value  (e.g.  3  per 
surveillance  period). 

d.  Quantity  of  Work.  Define  a  unit  (i.e.  single  occurrence)  of 
work  and  determine  the  number  of  units  to  be  performed  during  the 
surveillance  period. 

e.  Level  of  Surveillance.  If  more  than  one  level  of  surveillance 
is  desired,  it  is  recommended  that  three  carefully  defined  levels  of 
surveillance  be  identified  and  that  criteria  be  established  for  switching 
from  one  level  to  another.  These  surveillance  levels  are: 


-  Applied  to  good  but  not  exceptional  contractor 

performance.  This  level  of  surveillance  to  be  used 
when  contract  is  first  implemented. 
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Reduced  -  Applied  in  the  case  of  exceptional  contractor 
performance. 

Increased  -  Applied  in  the  case  of  poor  contractor  performance. 

f.  Sample  Size.  Determination  of  sample  size  for  planned  sampling 
is  subjective.  In  order  to  provide  consistent  surveillance,  the  rationale 
for  selecting  a  sample  size  must  be  identified.  Sample  size  will  vary 
depending  on  the  level  of  surveillance  used. 

g.  Sample  Selection  Criteria.  The  criteria  for  sample  selection 
must  be  documented  and  applied  consistently  from  surveillance  period  to 
surveillance  period.  If  there  is  no  consistency,  trends  in  contractor 
performance  cannot  be  detected. 

h.  Evaluation  Procedure.  List  any  procedures  or  checklists  used 
when  doing  inspection.  Evaluation  procedures  should  be  in  enough  detail  to 
allow  others  to  continue  the  same  manner  of  inspection  using  the  3aroe 
evaluation  criteria. 

i.  Analysis  of  Results.  The  Observed  Defect  Rate  (ODR)  for  a 
planned  inspection  is  the  total  number  of  defects  documented  during  the 
surveillance  period.  All  defects  detected  by  customer  complaints  and 
unschedule  inspections  are  included  in  this  total.  The  ODR  is: 

ODR  »  number  of  documented  defects. 

5.3.3  Random  Sampling.  This  method,  in  order  to  be  effective,  must  be 
properly  applied.  (Appendix  D  describes  the  mechanics  of  random  sampling.) 

a.  Contract  Requirement.  Contract  requirements,  identified  in  the 
Performance  Requirements  Summary  Table  (see  Figure  4-1),  are  listed  along 
with  the  performance  indicators  and  standards.  Specific  clauses  in  the  SOW 
establishing  these  requirements  are  listed. 

b.  Primary  Method  of  Surveillance.  Random  sampling  is  the  primary 
evaluation  method.  Validated  complaints  and  unscheduled  inspections  may  be 
considered  as  secondary  methods.  WARNING  information  collected  by  other 
surveillance  methods  can  never  be  combined  with  information  gathered  by 
landom  sampling.  Evaluation  results  collected  by  other  methods  serve  only 
as  supportive  data. 

c.  AQL.  For  evaluation  by  random  sampling,  AQL’s  are  stated  as  a 
percentage  and  predefined  as  5,  10,  15,  20  or  25  percent.  Sample  3izes  for 
AQL1 3  of  15,  20  and  25  percent  are  defined  in  the  Sample  Size  Tables  but  are 
not  generally  recommended. 

d.  Quantity  of  Work  Performed.  Define  a  unit  of  output  for  each 
service  that  is  subject  to  inspection.  It  is  important  to  accurately 
determine  the  quantity  of  work  performed  in  order  to  select  the  appropriate 
sample  size. 
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e.  Level  of  Surveillance.  The  level  of  surveillance  will  be  set  at 
one  of  three  levels,  and  the  level  of  surveillance  may  be  adjusted  monthly 
depending  on  the  Contractor's  performance. 

(1)  Initial  level  of  surveillance  is  normal  surveillance,  level 
II. 

(2)  Reduced  Surveillance.  If  contractor  performance  has  been 
"excellent,"  the  level  of  surveillance  could  be  reduced  to  level  I. 

(3)  Increased  surveillance.  If,  on  the  other  hand,  performance 
during  the  past  surveillance  period  was  poor,  surveillance  could  be 
increased  to  level  III. 


f.  Sample  Size.  Sample  size  for  random  sampling  is  determined  by 
use  of  tables  and  is  a  function  of  the  AQL,  quantity  of  work  performed  and 
level  of  surveillance.  {Sample  size  table  are  provided  in  Appendix  D.) 


g.  Sampling  Procedure.  To  assure  that  samples  are  selected 
completely  at  random,  a  random  number  table  mu3t  be  used.  (This  table  is 
provided  as  part  of  Appendix  E.) 


h.  Evaluation  Process.  List  any  procedures  or  checklists  used  when 
doing  the  inspection.  Evaluation  procedures  should  be  in  enough  detail  to 
allow  others  to  continue  the  same  manner  of  inspection  using  the  same 
evaluation  criteria. 


i.  Analysis  of  Results.  Analysis  for  random  sampling  will  consist 
of  computing  the  Observed  Defect  Rate  (ODR)  for  the  past  surveillance  period 


ODR 


Number  of  Defects 


Number  Evaluations  Conducted 


100 


The  number  of  defects  used  in  computing  the  ODR  is  derived  from 
the  samping  process.  Defects  detected  through  validated  complaints  or 
unscheduled  inspections  cannot  be  used. 


5.3.4  Validated  Customer  Complaints  A  Validated  Complaint  is  any  customer 


complaint  identifying  a  Contractor  defect  that  the  QAE  has  validated  by 
documentation  based  on  an  on-site  visit.  Complaints  not  validated  may  not 
be  used. 


a.  Contract  Requirement.  Contract  requirements,  identified  in  the 
Performance  Requirements  Summary  Table  (see  Figure  4-1) ,  are  listed  along 
with  the  performance  indicators  and  standards.  Specific  clauses  in  the  SOW 
establishing  these  requirements  are  listed. 


b.  Primary  Method  of  Surveillance.  The  primary  method  of 
surveillance  i3  validated  complaints.  A  secondary  evaluation  method  that 
may  be  used  is  unscheduled  inspections. 


c.  AQL,  AQL's  for  Validated  Complaints  should  be  stated  in  terms 
of  number  of  occurrences  per  surveillance  period. 


d.  Quantity  of  Work.  Although  the  quantity  of  work  will  not  have 
direct  effect  on  validated  complaint  evaluations,  this  information  is  useful 
when  putting  results  in  perspective. 

e.  Level  of  Surveillance.  This  is  not  applicable. 

f.  Sample  Size.  Tnis  is  not  applicable. 

g.  Sample  Selection  Criteria.  This  is  not  applicable. 

h.  Evaluation  Procedure.  Document  how  validation  of  complaints  i3 
to  be  performed.  Evaluation  procedures  should  be  in  enough  detail  to  allow 
others  to  continue  the  same  manner  of  inspection  using  the  same  evaluation 
criteria. 

i.  Analysis  of  Results.  Determine  the  number  of  validated 
complaints  for  the  past  surveillance  period.  If  there  is  a  good  customer 
complaint  program,  changes  in  the  number  of  complaints  per  surveillance 
period  may  be  useful  in  detecting  changes  in  the  Contractor's  overall  level 
of  performance.  The  Observed  Defect  Rate  (ODR)  for  validated  complaints  is: 

ODR  »  number  of  documented  defects 

5.3.5  Unscheduled  Inspections.  Unscheduled  inspections  do  not  have  formal 
QA  Plans.  This  type  of  surveillance  should  be  used  as  a  secondary,  or 
supportive,  surveillance  method  and  as  such,  be  subject  to  the  QA  Plan  of 
the  primary  method. 


CHAPTER  6 


SURVEILLANCE 


6.  GENERAL.  The  key  to  assuring  satisfactory  performance  from  service 

contracts  is  adequate  Government  surveillance  of  Contractor  performance. 
Hit-or-miss  surveillance  by  untrained  personnel  is  an  invitation  to  poor 
performance.  The  more  prone  (historically)  a  particular  type  of  work  i3  to 
shoddy  performance,  the  more  necessary  it  is  to  assign  an  adequate  number  of 
trained  and  qualified  personnel  (QAE's)  who  are  familiar  with  the  contract 
surveillance.  The  QAE  is  a  key  person  in  service  contract  management.  The 
QAE  serves  as  the  eyes  and  ears  of  the  SCM  and  as  such  must  demonstrate  a 
large  degree  of  common  sense  since  many  facets  of  the  job  are  subjective  and 
open  to  criticism.  NAVFAC  Quality  Assurance  Evaluator  (QAE)  Training 
Manual,  MO-326.2,  provides  in-depth  information  on  QAE  duties  and  the 
surveillance  process. 

6.1  SURVEILLANCE  PROCESS .  The  surveillance  process  is  a  system  composed 
of  many  key  elements.  Figure  6-1  illustrates  this  system. 

6.1.1  Inputs.  The  key  input  to  surveillance  is  contract  requirements. 
These  requirements  dictate  what  work  the  Contractor  is  to  perform  and  what 
the  QAE  is  to  evaluate.  Next  in  importance  is  the  Contractor's  work 
schedule.  The  schedule  is  necessary  in  order  for  the  QAE  to  know  when  work, 
which  is  not  scheduled  by  contract  requirements,  is  to  be  performed.  The 
intensity  of  surveillance  is  influenced  to  a  degree  by  the  Contractor's  past 
performance.  During  the  surveillance  period  the  number  and  type  of  customer 
complaints  received  will  affect  the  QAE's  schedule. 

6.1.2  Process.  The  process,  as  displayed  in  Figure  6-1,  may  be  divided 
into  four  main  part3:  planning  for  surveillance,  conducting  surveillance, 
analysis  of  surveillance  results,  and  taking  action  based  on  those  results. 

a.  Planning  for  surveillance  includes  QA  Plans  and  Monthly  QAE 
Schedules.  QA  Plans  are  developed  prior  to  contract  award  and  in  most  cases 
remain  unaltered  throughout  the  life  of  the  contract.  Monthly  QAE 
Surveillance  Schedules  are  developed,  based  on  QA  Plans,  at  the  start  of 
each  surveillance  period  (a  period  is  usually  one  month). 

b.  Contract  surveillance  is  conducted  in  accordance  with  QA  Plans 
and  the  QA  Monthly  Schedule.  If,  during  the  surveillance  period,  major 
discrepancies  are  noted  (and  documented) ,  t*e  QAR  will  alert  the  SCM  that 
action  should  be  taken. 

c.  At  the  end  of  the  surveillance  period,  "documented"  surveillance 
results  are  analyzed  to  determine  the  Contractor's  overall  level  of 
performance. 


d.  Based  on  the  Contractor's  performance,  there  are  several  courses 
of  action  that  may  be  taken.  First,  deductions  will  be  made  for  all 
observed  and  documented  cases  of  non-compliance,  regardless  of  the 
Contractor's  overall  level  of  performance.  Other  specific  actions  that  may 
be  taken  include:  "jaw  boning"  the  Contractor;  issuance  of  a  Contract 
Discrepancy  Report  (CDR) ,  Cure  Notice  or  Show  Cause  Notice;  and  contract 
termination.  The  person  taking  action  may  be  the  SCM,  ROIC,  OIC,  or 
Commander,  NAVFAC.  Regardless  of  the  course  of  action,  GOOD  DOCUMENTATION 
IS  REQUIRED. 
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FICURE  6-1 

Contract  Surveillance  Process 


6.2  QUALITY  ASSURANCE  EVALUATOR  (QAE) .  QAE's  must  have  qualifications 
in  both  the  technical  aspects  of  the  contracted  function  and  contract 
surveillance  methods. 

a.  Technical  expertise  for  evaluating  work  quality  is  found  within 
the  organizational  component  that  would  be  responsible  for  work 
accomplishment  if  it  were  to  be  performed  by  Government  forces.  The  QAE 
should  be  drawn  from  this  component.  (In  the  case  of  CA  conversions  the 
activity  will  retain  4  percent  of  the  organizational  component  being 
converted  to  contract  performance.  This  4  percent  i3  intended  for  contract 
management  -  i.e.  SCM  and  QAE  duties.) 

b.  The  TRCO  is  the  person  usually  responsible  for  selecting  QAE' 3. 
Once  selected  for  QAE  duties,  candidate  QAE's  must  be  trained  in  contract 
surveillance  methods.  This  training  is  available  through  the  QAE  Training 
Course  offered  periodically  by  NAVFAC  Engineering  Field  Divisions  (EFD’s). 

6.2.1  Authority.  QAE  derives  his  authority  by  delegation  from  the  SCM  and 
acts  as  his  representative  when  authorized.  In  no  case  can  the  SCM  delegate 
authority  to  the  QAE  that  he  himself  does  not  have. 

a.  Delegation  of  authority  is  by  letter  from  the  SCM  to  the  QAE. 

b.  The  QAE  has  no  authority  to  allow  deviations  from  essential 
contract  requirements,  but  when  authorized  by  hi3  superior,  he  may  approve 
minor  deviations  not  involving  change  in  contract  time,  price,  or  basic 
design. 


c.  I7ie  QAE  has  no  authority  to  direct  or  interfere  with  the  methods 
of  performance  by  the  Contractor  or  to  issue  instructions  directly  to  any  of 
the  Contractor's  personnel  unless  the  methods  being  used  are  unsafe. 

6.2.2  Responsibilities.  The  QAE,  because  of  his  familiarity  with  the 
contract,  the  Contractor,  and  the  customer,  is  involved  with  several  aspects 
of  Service  Contracting. 

a.  Specific  QAE  responsibilities  will  depend  upon  local  conditions, 
sizt  of  the  contract,  QAE  collateral  duties,  etc.  Assigned  responsibilities 
should  be  agreed  upon  by  the  SCM,  the  QAE,  the  TRCO,  and  other  interested 
parties  Defore  the  contract  start  date.  Assigned  QAE  responsibilities  should 
be  3tated  in  writing. 

b.  Specific  QAE  duties  are  to: 

o  Review  plans  and  specifications  prior  to  IFB/RFP; 
o  Conduct  Contractor  pre-bid  site  visits? 
o  Assist  in  pre-award  surveys; 

Review  Contractor  schedules  and  advise  SCM  of  acceptability; 


o 
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o  Attend  pre-bid  and  pre-start  conferences; 

o  Coordinate  provision  of  Government  furnished  space,  utili¬ 
ties,  equipment  and  material; 

o  Prepare  Quality  Assurance  Plans; 

o  Prepare  surveillance  schedules; 

o  Perform  surveillance  and  document  Contractor's  performance; 

o  Conduct  surveillance  on  accomplishment  of  re-performed  work; 

o  Monitor  labor,  safety  and  security  practices,  and  document 
results; 

o  Recommend  payment  deductions  for  unsatisfactory  work  to 
SCM;  and 

o  Assist  in  the  preparation  of  Government  estimate  for  change 
orders. 

6.2.3  STAFFING.  An  adequate  level  of  QAE  staffing  is  required  to  make  any 
QA  program  work  well.  There  are  two  approaches  to  staffing  for  contract 
surveillance:  (a)  write  the  QA  program  to  accommodate  the  number  of  QAE's 
currently  available,  or  (b)  to  write  the  QA  program  to  provide  the  desired 
level  of  surveillance  and  staff  to  that  program.  Obviously  the  latter  is 
the  preferred  approach.  The  chief  problem  i3  converting  the  desired  level 
of  surveillance  into  manhour  requirements.  QA  Plans,  and  the  subsequent  QAE 
Schedules,  provide  a  means  of  determining  QAE  manhour  requirements.  These 
documents,  if  properly  prepared,  will  identify  QAE  staffing  requirements  for 
contract  surveillance. 

6.3  QAE  SCHEDULE .  The  development  of  an  effective  evaluation  schedule 
should  be  of  the  utmost  importance  to  the  QAE.  The  evaluation  schedule 
allows  the  QAE  to  plot  where  he  should  be  on  any  given  day  of  the  week.  By 
developing  a  balanced  Inspection  schedule,  a  QAE  can  be  much  more  effective 
in  his  job.  It  allows  the  QAE  to  plan  hi3  workday  in  advance  to  the  best 
advantage.  It  also  allows  him  to  utilize  his  time  and  eliminate  some 
potentially  wasteful  actions  (for  example,  excessive  travel  time  between 
inspections).  QAE  Schedules  serve  three  purposes, they  are: 

a.  Optimizing  Time.  The  QAE  will  use  his  established  schedule  to 
plan  his  work.  By  making  maximum  use  of  a  good  schedule,  the  QAE  will 
optimize  use  of  his  time. 

b.  Management  Control.  The  SCM  is  provided  a  copy  of  each  QAE 
Schedule.  The  SCM  has  the  responsibility  to  see  that  surveillance  of 
service  contracts  is  properly  conducted.  The  QAE  Schedule  provides  him  the 
information  necessary  to  monitor  the  QA  program. 
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c.  Audit.  The  QAE  Schedule,  along  with  completed  evaluation 
reports,  provide  an  audit  trail  for  contract  surveillance.  The  established 
schedule  as  updated  during  execution,  should  reflect  what  was  actually 
accomplished. 

6.3.1  Schedule  Development.  QAE  schedules  are  based  on  established  QA 
Plans.  When  developing  monthly  schedules,  the  QAE  will  use  the  evaluation 
worksheets  developed  for  that  month.  A3  he  prepares  his  schedule  he  may 
find  it  convenient  to  combine  surveillance  requirements  in  order  to 
streamline  the  daily  schedule.  When  developing  schedules  it  may  be 
necessary  to  modify  QA  Plans  in  order  to  achieve  the  most  effective 
allocation  of  QAE  time.  However,  plans  based  on  random  sampling  CANNOT  be 
modified  in  this  manner. 

a.  The  Schedule.  An  example  of  a  QAE's  Schedule  is  shown  in  Figure 
6-2.  This  example  shows  only  a  6-day  schedule.  The  QAE  must  make  up  enough 
sheets  to  include  each  day  of  the  month.  Along  the  top  of  the  schedule, 
insert  the  items  to  be  checked  during  the  month.  Along  the  bottom  of  the 
schedule,  indicate  the  number  of  observations  to  be  made  during  the  month. 

b.  Filling  In  and  Updating  the  Schedule.  To  fill  in  the  inspection 
schedule,  the  QAE  refers  to  the  QA  Plans  for  each  service  being  monitored. 
The  QA  Plan  is  used  to  determine  the  inspections  (observations)  to  be  made 
during  the  month. 

(1)  Contract  surveillance  must  cover  all  hours  of  operation. 
Random  observations  are  scheduled  at  night,  on  weekends,  and  on  holidays 
when  services  are  performed  during  these  periods.  Areas  that  are  monitored 
on  a  set  schedule  are  included  in  the  monthly  schedule.  This  monthly 
schedule  shows  where  and  what  the  QAE  is  monitoring  at  all  times. 

(2)  Post  any  changes  to  the  schedule  weekly,  and  send  copies  to 
the  SCM  and  to  the  TRCO.  Document  and  explain  the  reasons  for  each  change. 
Actual  surveillance  activity  recorded  on  the  evaluation  worksheets  must  be 
comparable  to  the  monthly  schedule. 

(3)  As  updated,  one  must  be  able  to  conduct  a  complete  audit 
trail  from  the  monthly  schedule,  to  observing  the  QAE  perform  sampling,  to 
completion  of  the  evaluation  worksheet. 

(4)  After  the  schedule  is  completed  and  filled  in,  thi3  form  is 
"FOR  OFFICIAL  USE  ONLY"  and  must  not  be  shown  to  the  Contractor. 

c.  Review  and  Approval.  The  QAE's  Schedule  must  be  submitted  to 
the  SCM  for  review  and  approval.  The  QAE  is  responsible  for  posting 
changes,  as  they  occur,  to  the  schedule  throughout  the  month.  This  schedule 
becomes  a  formal  part  of  the  surveillance  documentation,  and,  as  such,  it 
must  be  auditable. 

6.4  SURVEILLANCE.  Contract  surveillance  involves  using  the  QA  Plan 
called  for  in  the  monthly  schedule. 


6.4.1  Performance  of  Surveillance ■  Contract  surveillance  is  performed  in 
accordance  with  the  QA  Plan:  method  of  surveillance,  sample  size/selection, 
and  evaluation  critieria  are  specified  in  this  plan. 

a.  The  QA  program  must  provide  an  adequate  and  consistent  level  of 
surveillance.  It  is  important  that  QA  plan  be  followed  and  deviations 
documented  in  order  that  this  program  can  be  audited. 

b.  Timing  of  Inspections: 

(1)  In  some  cases  inspection  will  have  to  be  conducted  during 
the  period  of  work  performance.  For  example: 

-  When  the  Contractor  is  performing  maintenance  on  a  piece 
of  equipment,  the  QAE  may  have  to  inspect  the  work  before  the  equipment  is 
fully  reassembled. 


-  If  work  on  a  building's  electrical  or  plumbing  systems  l« 
performed,  the  QAE  would  have  to  inspect  before  siding  or  dry  wall  is 
replaced. 


-  To  determine  if  the  Contractor  is  maintaining  a  base 
shuttle  bus  schedule,  the  QAE  must  be  at  the  stop  at  the  scheduled  time. 

(2)  For  daily  services,  such  as  custodial  services,  the  QAE 
should  conduct  inspections  shortly  after  work  performance,  but  prior  to 
occupant  use. 


(3)  In  many  cases  services  performed  will  provide  outputs  of  a 
lasting  nature  and  may  be  inspected  days  after  actual  performance.  Work, 
such  as  painting,  resurfacing  of  roads,  glass  replacement,  tree  pruning, 
etc.,  are  examples  of  this  type  of  work. 

(4)  Some  services  performed  by  the  Contractor  may  be 
inspectable  at  any  time;  for  example:  if  a  grounds  maintenance  contract 
requires  a  level  of  maintenance  (vs.  frequency  of  work) ,  the  QAE  will  be 
monitoring  the  condition  of  the  grounds  rather  then  work  performed  (e ,g. 
grass  to  be  between  2  and  4  inches  in  height.)  A  watch  standing  requirement 
such  as  guard  service  or  fire  protection  requires  Contractor  personnel  on 
duty  24  hours  a  day. 

c.  It  is  good  practice  to  make  surveillance  findings,  good  or  bad, 
available  to  the  Contractor  on  a  daily  basis.  Provided  information  does  not 
relieve  the  Contractor  of  his  Quality  Assurance  efforts  but  is  intended  to 
keep  the  Contractor  advised  of  the  Government's  perception  of  the  quality  of 
performed  work. 

6.4.2  Documentat ion .  Just  as  services  required  of  the  Contractor  have 
outputs,  Government  surveillance  has  outputs.  One  of  the  key  outputs  is 
documentation,  which  consists  of:  the  QAE's  monthly  schedule,  completed 
evaluation  worksheets,  records  of  customer  complaints,  and  any  other 
material  that  reflects  the  quaiity/quantity  of  Contractor  performance. 


QAE: 


SCHEDULE  FOR  WEEK  OF  :  Jane  22  thru  27 


I 

I 

I 


DAY 


TIME 

MON 

TUB 

WEN 

THU 

FRI 

0700 

P-  INSP 
MESS 

HALL  #2 

P-INSP 

MESS  HALL 

#4 

R-INSP 

BLDG.  1,9 
13,20,31 

0800 

Meet 

with  SCM, 

MONITOR 

TRASH 

COLLECTION 

R-INSP 
BLDG.  11, 
14,18,20 

MONITOR 

TRASH 

0900 

P-INSP 

CONT. 

QC  FILE 

ROIC,  AND 
CONT.  REP 
TO  DISCUSS 

IN  RES. 

AREA  A 

COLLECTION 

IN  RES. 

AREA  B 

1000 

CHANGE 

ORDERS 

P-INSP 

WASH 

RACK 

1100 

1200 

LUNCH 

LUNCH 

LUNCH 

LUNCH 

1300 

PAPER 

PINSP 

SANITARY 

LUNCH 

1400 

WORK 

IN 

OFFICE 

LANDFILL 

OPERATION 

MEET 

1500 

R-INSP 
BLDG.  1 
5,32,41 

R-INSP 
AREA  5,9 
11,15 

WITH 

SCM 

1600 

R-INSP 

AREAS  3,5, 
7,17,19 

P-INSP 

COMMISARY 

P-INSP 

MESS  HALL 

#2 

1700 

1800 

1900 

: 

NOTE:  P-INSP  =  Planned  Inspections 

R- INST  *  Random  Inspections 


FIGURE  6-2 

QAE  lurveillance  Schedule 
(Sample) 


a.  Recording  Observations.  Evaluation  Worksheets  are  used  for  each 
QA  Plan.  They  are  used  to  tally  information  on  scheduled  observations  and 
noted  defects.  Each  observation  in  the  sample  is  recorded  on  the  Worksheet, 
and  the  Worksheets  then  become  a  formal  Government  record. 

b.  Potential  Unsatisfactory  Performance.  If  surveillance  indicates 
that  the  number  of  defects  for  the  month  may  exceed  the  AQL,  the  QAE  should 
try  to  identify  and  document  the  cause  of  the  problem. 

(1)  The  cause  of  the  problem  may  lie  with  the  Government. 

(a)  Is  delivery  of  Government-provided  material  or 
equipment  the  problem? 

(b)  Are  Government  employees  (civilian  or  military) 
disrupting  the  Contractor's  work  efforts? 

(2)  If  the  cause  of  the  problem  lies  with  the  Contractor,  the 
QAE  should  determine  why. 

(a)  Does  the  Contractor  have  an  adequate  number  of  people, 
or  properly  trained  people,  at  the  work  site? 

(b)  Is  Contractor  work  supervision  adequate? 

(c)  Is  the  Contractor’s  QC  program  identifying  these 

problems?  (It  should  be.)  * 

(d)  Are  the  proper  equipment  and  materials  being  used? 

(e)  Is  the  work  method  used  capable  of  producing  the 

required  output? 

c.  During  the  course  of  the  contract  the  QAE  retains  a  copy  of  all 
QAE  Schedules,  Evaluation  Worksheets,  and  checklists.  At  the  end  cf  the 
contract  period,  the  QAE  for.’rds  these  records  for  inclusion  in  one 
contract  file.  However,  when  a  specific  service  becomes  unsatisfactory 
during  a  surveillance  period,  a  copy  of  the  inspection  documentation 
supporting  the  contract  discrepancy  is  forwarded  to  the  S CM/ROIC  tor  action. 

d.  As  mentioned  above  it  is  good  practice  to  keep  the  Contractor 
appraised  of  surveillance  results.  One  way  to  do  this  is  to  provide  the 
Contractor's  Representative  a  copy  of  the  Evaluation  'Worksheet.  It  is 
recommended  that  Contractor's  Representative  initial  the  original  Evaluation 
Worksheet  indicating  that  he  has  received  a  copy. 

6.5  SURVEILLANCE  RESULT5.  It  is  the  QAE' 3  duty  to  make  QA  evaluation 

results  known  to  the  SCM  who  then  is  responsible  for  taxing  the  appropriate 
action.  At  the  end  of  the  month  the  QAE  will  tally  the  results  for  all 
Evaluation  Worksheets,  checklists,  etc.,  to  determine  the  Contractor’s 
overall  performance  with  respect  to  each  QA  Plan. 
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6.5.1  3xcellent  Performance.  If  the  Contractor  has  performed  in  the  best 
possible  manner  and  there  were  few  defects  noted  (Observed  Defect  Rate  (ODR) 
less  than  1/2  the  AQL) ,  the  QAE  might  suggest  that: 

a.  The  Contractor  should  be  notified  by  the  SCM  that  he  is 
performing  satisfactorily. 

b.  The  level  of  surveillance  might  be  reduced. 

c.  Deductions  will  be  made  on  all  documented  defects. 

6.5.2  Good  Performance.  When  a  Contractor's  quality  control  program 
works,  good  performance  results.  If  the  result  of  a  QAZ's  surveillance 
shows  consistently  good  performance,  the  amount  of  surveillance  can  be 
decreased. 

a.  Deductions  will  be  made  on  all  documented  defects. 

b.  Reduced  Surveillance.  Inspection  can  be  reduced  when  the 
following  conditions  have  been  met  for  a  surveillance  period. 

(1)  The  preceding  month's  work  (or  number  of  months  as 
specified  in  the  QA  Plan)  has  been  acceptable. 

(2)  The  percentage  of  defects  in  the  preceding  month (3)  is  less 
than  one  half  of  the  AQL. 

(3)  The  normal  sample  size  is  being  used. 

(4)  The  TRCO  and  the  SCM  agree  to  use  reduced  inspection. 

c.  Returning  to  Normal  Surveillance.  When  reduced  surveillance  is 
in  effect,  return  to  normal  inspection  the  next  month  under  the  following 
conditions: 

(1)  When  the  percentage  of  defects  exceeds  the  AQL  under 
reduced  sampling,  or 

(2)  The  TRCO  and  the  SCM  deem  it  necessary  to  return  to  normal 

inspection. 

6.5.3  Questionable  Performance.  An  outcome  of  questionable  performance 
can  only  result  when  random  sampling  is  the  surveillance  method  used. 

a.  Random  sampling  procedures  take  into  consideration  potential 
errors  in  results.  Since  random  sampling  only  provides  an  estimate  of  the 
true  defect  rate,  a  margin  for  error  must  be  used.  This  is  done  by 
specifying  the  accuracy  desired  of  the  ODR,  as  compared  to  the  true  defect 
rate.  Accuracy  is  defined  to  be  one  half  of  the  AQL.  If,  for  example,  the 
AQL  was  10  percent,  accuracy  would  be  5  percent.  It  is  this  gray  area, 
where  the  ODR  falls  between  the  AQL  and  1.5  times  the  AQL,  that  results  are 
questionable. 


b.  The  recommended  actions  for  questionable  performance  are: 


(1)  Deduct  for  all  documented  defects. 

(2)  If  there  are  a  significant  number  of  validated  customer 
complaints  and/or  defects  detected  by  unscheduled  inspections,  issue  a 
Contract  Discrepancy  Report  (CDR) . 

(3)  Go  to  an  increased  level  of  evaluation. 

6.5.4  Poor  Performance.  If  the  Contractor  has  displayed  poor  performance 
(l.e. .  the  ODR  exceeds  the  AQL,  or  for  random  sampling  the  ODR  exceeds  1.5 
times  the  AQL) ,  then  the  following  are  the  actions  normally  taken  in 
addition  to  deductions. 

a.  If  the  QAE  first  determines  that  the  Government  created  any  of 
the  discrepancies,  these  are  not  to  be  counted  against  the  Contractor's 
performance.  When  the  Government  has  caused  the  Contractor  to  perform  in  an 
unsatisfactory  manner,  the  QAE  prepares  a  letter  to  be  3ent  to  the 
responsible  organization,  requesting  corrective  action.  The  QAE  sends  it  to 
the  responsible  organization  through  the  SCM. 

b.  If  the  Government  did  not  cause  the  discrepancy,  the  QAE  tells 
the  Contractor's  site  manager,  in  person,  when  discrepancies  occur  and  asks 
the  Contractor  to  correct  the  problem.  The  QAE  makes  a  notation  on  the 
Evaluation  worksheet  of  the  date  and  time  the  deficiency  was  discovered  and 
has  a  Contractor's  representative  initial  the  entry  on  the  checklist. 

c.  Increased  Surveillance.  The  level  of  surveillance  can  be 
increased  when  the  following  conditions  have  been  met  for  a  sampling  period: 

(1)  The  preceding  surveillance  period  (last  month's  inspection) 
has  been  unsatisfactory  (ODR  exceeds  AQL) . 

(2)  Normal  sample  size  is  being  used. 

(3)  The  TRCO  and  the  SCM  agree  to  increased  inspection. 

(4)  Use  the  Sample  Size  shown  in  Table  III,  Appendix  E,  (or  go 
to  100  percent  inspection) 

(5)  Return  to  Normal  Sample  Size  if  after  one  month  the  ODR  is 
less  than  the  AQL. 

d.  When  the  Contractor  is  responsible  for  failing  to  meet  the 
limits  of  satisfactory  performance  (the  AQL) ,  the  SCM  issues  a  Contract 
Discrepancy  Report  (CDR)  to  the  Contractor  (discussed  in  6.S.2  below).  If 
the  failure  is  serious  enough,  issue  the  CDR  at  the  time  of  the 
unsatisfactory  performance  rather  than  at  the  end  of  the  month. 

e.  If  the  Contractor  does  not  achieve  satisfactory  performance  in 
that  specific  service  by  the  end  of  the  next  month,  the  SCM  issues  another 
CDR,  and  the  ROIC  may  call  in  the  Contractor  Representative  for  a  personal 
review  of  the  problem. 
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f.  If  a  third  CDR  must  be  issued,  the  ROIC  should  consider  Issuing 
a  cure  notice.  (However,  a  cure  notice  can  be  issued  sooner,  if  necessary.) 


g.  Depending  on  the  Contractor's  overall  performance,  the 
Government  may  issue  a  Show  Cause  letter  if  the  reply  to  a  cure  notice  is 
unsatisfactory.  (EFD  02  approval  is  required.) 

6.6  TARING  ACTION.  The  QAE  may  check  the  Contractor's  performance  and 
document  any  non-compliance,  but  only  the  OIC  may  take  formal  action  against 
the  Contractor  for  unsatisfactory  performance.  This  section  lists  the 
normal  steps  to  be  taken  by  contract  administration  when  the  QAE  reports 
these  deficiencies.  The  actions  listed  are  not  hard-and-fast  rules  and 
represent  a  minimum  level  of  action.  More  serious  action  can  be  taken 
sooner. 


6.6.1  Deductions.  NAVFAC  policy  is  that  deductions  will  be  made  for  each 
observed  and  documented  defect.  Extrapolated  deductions  based  on  random 
sampling  will  not  be  used. 

a.  The  QAE  makes  a  recommendation  on  the  amount  of  payment 
deductions  to  be  made  based  on  documented  deficiencies  multiplied  by  the 
price  shown  in  the  Schedule  of  Deductions,  or  in  the  Items  of  Rid  for 
indefinite  quantity  work  items,  and  the  amount  of  liquidated  damages  shown 
in  the  "Consequences  of  Contractor's  Failure  to  Perform"  clause. 

b.  When  the  Contractor's  performance  is  unsatisfactory,  l.e. , 
exceeds  the  AQL  as  defined  in  the  QA  Plan  and  a  formal  action  is  indicated, 
the  QAE,  the  TRCO,  the  SCM,  the  ROIC,  and  the  Contract  Specialist  meet  to 
determine  what  action  is  appropriate. 

c.  If  a  decision  is  reached  to  not  take  a  monetary  deduction,  the 
reasons  are  to  be  documented.  The  ROIC  must  indicate  agreement  with  the 
decision  by  signing  the  decision  documentation. 

d.  Example  Deductions: 

(1)  Example  1.  The  QAE  has  conducted  surveillance  of  a 
contract  requirement  using  planned  sampling  supported  by  validated  customer 
complaints.  At  the  end  of  the  month  results  ares 

o  Number  of  inspections  conducted  -  100 

o  Number  of  defects  found  through  planned  inspection  -  3 

o  Number  of  customer  complaints  -  15 

o  Number  of  complaints  validated  -  11 

(a)  The  Contractor  was  not  given  the  opportunity  to  per¬ 
form  the  work  due  to  time  constraints. 

(b)  Observed  Defect  Rate  (ODR)  is  14  (3  +  11).  Assuming 
AQL  was  10  de facts/month,  the  Contractor  performance  is  POOR.  A  CDR  should 
be  Issued. 


(c)  The  established  price  for  each  unit  of  work  is  $10 
(taken  from  the  Schedule  of  Deductions) .  Recommended  deduction  is  $140  plus 
an  additional  10  percent  for  administrative  costs  which  an  allowed  under 
the  "CONTRACTOR ' s  CONSEQUENCES  OF  FAILURE  TO  PERFORM"  clause  of  Section 
00004.  Total  amount  recommended  for  deduction  is  $154. 

(2)  Example  2.  Random  sampling  has  been  the  method  used  to 
monitor  the  Contractor's  performance.  Results  at  the  end  of  the  month  are: 

o  Number  of  inspections  conducted  -  37 

o  Number  of  defects  found  through  random  sampling  -  6 

o  Number  of  customer  complaints  -  5 

o  Number  of  complaints  validated  -  2 

o  Number  of  defects  documented 

by  Unscheduled  Inspections  -  2 

(a)  The  Contractor  was  not  given  the  opportunity  to 
perform  the  work  due  to  time  constraints. 

(b)  The  ODR  is  6.9  percent  (6  divided  by  87).  NOTE:  The 
2  defects  identified  by  customer  complaints  and  the  2  day  unscheduled 
inspections  are  not  included  in  the  ODR  computation. 

(c)  The  established  price  for  each  work  occurrence  is  $15. 
Recommended  deduction  is  $150  ($15  X  10  defects;  all  defects  are  used  for 
deduction  calculations) ,  plus  $15  for  administrative  costs.  (The 
"CONSEQUENCES  OF  CONTRACTOR'S  FAILURE  TO  PERFORM"  Clause,  Section  00003 
allows  an  additional  10  percent  for  defective  work  that  is  either 
re-performed  by  the  Contractor  or  left  unperformed.)  Total  amount 
recommended  for  deduction  is  $165. 

(3)  Example  3.  The  QAE  has  used  100  percent  inspection  to 
evaluate  the  Contractor's  performance.  At  the  end  of  the  month  results  are: 

o  Number  of  inspections  conducted  -  85 

o  Number  of  defects  found  -  7 

o  Number  of  defects  corrected  by 
Contractor  -  4 

o  Number  of  defects  corrected  by  Government 
Employees  -  3 

(a)  The  nature  of  required  work  was  such  that  it  could  be 
re-performed,  and  it  was  necessary  that  all  work  be  accomplished  during  the 
month. 


(b)  The  ODR  is  7.  Since  this  work  was  very  important  to 
the  activity'  mission  the  AQL  was  set  at  zero.  The  Contractor's  overall 
performance  is  POOR.  A  CDR  should  be  issued. 
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(c)  The  established  price  per  unit  of  work  is  $500.  The 
recommended  deduction  is  $2000  based  on  the  following: 

-  Work  re-per formed  by  the  Contractor  is  subject  to  a 
10  percent  deduction  for  administrative  costs  as  specified  in  the 
■CONSEQUENCES  OF  CONTRACTOR'S  FAILURE  TO  PERFORM*  clause,  Section  00003. 

(In  this  case  administrative  cost  is  taken  to  mean  the  cost  of 
re- inspection;  therefore,  the  Government  must  be  able  to  show  that 
re-inspection  was  performed.)  This  amounts  to  $200  ($500  x  4  x  10  percent). 

-  Work  performed  by  Government  employees  is  subject 
to  20  percent  deduction  to  reflect  the  cost  of  the  liquidated  damages  as 
specified  in  the  "CONSEQUENCES  OF  CONTRACTOR'S  FAILURE  TO  PERFORM"  clause. 
Section  00003.  If  the  work  in  question  was  bid  lump  sum,  and  the 
Contractor's  invoice  reflects  that  work,  the  cost  of  that  work  must  be 
deducted.  This  amounts  to  $1300:  $300  ($500  X  3  x  20  percent)  plus  $1500 
($500  X  3)  for  the  cost  of  the  work.  (NOTE:  If  the  work  in  question  was 
bid  on  an  indefinite  quantity  work  item  and  the  Contractor's  invoice  did  not 
reflect  the  work  in  question,  only  $800  would  be  deducted  for  that  work 
performed  by  Government  employees.) 

6.6.2  Contract  Discrepancy  Report  (CDR) .  CDR's  are  written  to  identify 
documented  cases  of  poor  Contractor  performance.  The  Contractor,  upon 
receiving  a  CDR,  must  identify,  in  writing,  how  future  occurrences  of  the 
problem  will  be  prevented  (i.e. .  the  corrective  action  he  intends  to  take). 
Based  on  the  Contractor's  response,  the  Government  may  or  may  not  take 
further  action. 

a.  When  the  Contractor's  overall  performance  for  any  given  contract 
requirement  is  classified  as  "poor",  the  QAE  should  recommend  to  the  SCM 
that  a  CDR  (Figure  6-3)  be  issued. 

b.  The  QAE  is  responsible  for  identifying  the  problem  that  caused 
the  poor  performance.  This  information  is  not  relayed  to  the  Contractor. 

The  QAE  will  use  this  information 'to  evaluate  the  Contractor's  response. 

The  Contractor  should  have  a  Quality  Control  Program  that  gives  him  feedback 
on  his  performance. 

c.  If  the  Contractor '3  response  is  likely  to  correct  the  problem:, 
the  QAE  should  recommend  to  the  SCM  that  further  Government  action  is  not 
required  with  the  exception  of  an  increased  level  of  surveillance.  If  the 
response  is  not  likely  to  correct  the  problem,  the  QAE  should  identify  why 
it  is  not  adequate  and  recommend  further  action  to  be  taken  by  the 
Government. 
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TO:  (Contractor  &  Manager's  Name) 


3.  FROM:  (Name  of  SCM/ROIC) 


4.  DISCREPANCY  OR  PROBLEM:  (Describe  in  detail:  Include  reference  to 
SOW  Directive:  Attach  continuation  sheet  if  necessary.) 


5.  SIGNATURE:  (SCM/ROIC) 

6.  DATE: 

7.  TO:  (Contracting  Officer) 

FROM:  (Contractor) 

8.  CONTRACTOR  RESPONSE:  (Contractor's  proposed  solution  to  correct  future 
occurrences  of  the  problem,  use  Continuation  Sheet  if  necessary) 


9.  SIGNATURE:  (Contractor  Representative) 


DATE: 


10.  GOVERNMENT  EVALUATION:  (Is  the  Contractor's  response  a  viable  solution 
to  the  problem?) 


11.  GOVERNMENT  ACTIONS:  (Does  the  Government  accept,  propose  modification, 
or  reject  the  Contractor's  response?) 


12.  CLOSE  OUT: 

QAE  Signature:  Date: 
SCM  Signature:  Date: 
ROIC  Signature:  Date: 


SAMPLE  QUALITY  ASSURANCE  (QA)  PLJ 
QA  PLAN  TOR  SCHEDULED  INDUSTRIAL  SOLID  WASTE  COLLECTION 


1.  CONTRACT  REQUIRBffitTT.  Clause  3.3  of  Section  00005  specifies  the  re¬ 
quirements  for  scheduled  industrial  solid  waste  collection.  The  contractor 
is  required  to  establish  collection  schedules.  Collection  standards  are: 


Scheduled  pick-ups  are  made  within  four  hours  of  the  established 
time. 

Pick-ups  are  made  outside  of  normal  rush  hours  (or  other  site  sped 
f  ic  time) . 


Noise  levels  not  to  exceed  90  dBA  as  measured  on  a  general  purpose 
sound  level  meter,  (ANSI  standard  SI. 4-1961). 

No  more  then  five  pieces  of  debris  within  15  yard  radius  of  con¬ 
tainer  after  pick-up. 

Containers  re-positioned  within  five  feet  of  specified  location. 


2.  PRIMARY  METHOD  OF  SURVEILLANCE, 
complaints. 

3.  ACCEPTABLE  QUALITY  LEVEL  (AQL) . 


Random  sampling  supported  by  customer 
10  percent. 


4.  QUANTITY  OP  WORK.  Appendix  B  to  the  contract  specifies  85  sites  for 
scheduled  industrial  solid  waste  collection.  The  contractor  supplied  col¬ 
lection  schedule  specifies  50  of  these  sites  for  weekly  collection  and  the 
remaining  35  sites  on  a  twice  weekly  schedule.  This  gives  a  total  of  480 
work  occurrences  per  month  ((50  X  4)  +  (35  X  4  X  2)  *  480). 


5.  LEVEL  OF  SURVEILLANCE.  Normal  surveillance  (Level  II)  is  recommended 
initially.  If  contractor  performance  i3  good  (i.e. ,  Observed  Defect  Rate 
(ODR)  is  1/2  the  AQL,  or  less) ,  for  2  months  in  a  row,  then  reduced  surveil¬ 
lance  (Level  I)  should  be  considered. 


If  the  Contractor’s  performance  is  questionable  for  the  past  month  (i_ 
ODR  greater  than  the  AQL  but  less  than  or  equal  to  1.5  times  the  AQL)  and 
surveillance  was  used,  return  to  normal  surveillance. 


If  Contractor  performance  is  unsatisfactory  in  any  month  (i.e. ,  ODR  is 
greater  than  1.5  times  the  AQL),  then  increased  surveillance  (Level  III) 
should  be  implemented  for  the  following  month. 

6.  SAMPLE  SIZE.  Using  the  Sample  Size  Tables,  the  following  sample  3izes 
are  required  for  an  AQL  of  10  percent  and  a  population  of  480: 


LEVEL  OF  SURVEILLANCE 

NO.  PER  MONTH 

NO.  PER  WEEK* 

Normal  (Level  II) 

109 

28 

Reduced  (Level  I) 

34 

9 

Increased  (Level  III) 

•  178 

45 

‘Assumes  four  weeks  per  month 


7.  SAMPLE  SELECTION  PROCEDURE.  Since  random  sampling  is  the  method  of 
evaluation  to  be  used,  sample  selection  will  be  by  a  random  process.  The 
selection  process  is  as  follows: 

a.  Collection  sites  are  numbered  from  1  to  120. 

b.  On  a  monthly  basis  select  four  sets  of  random  numbers  of  sizes  in¬ 
dicated  above  ti.e. ,  28  for  Level  II  surveillance) .  Each  set  of  numbers  may 
have  duplicate  numbers  selected  for  those  sites  on  a  twice  weekly  collection 
schedule.  Numbers  may  be  duplicated  between  sets. 

c.  Each  set  of  numbers  selected  will  be  matched  against  the  numbered 
sites.  These  sites  are  the  collection  sites  to  be  monitored  for  each  re¬ 
spective  week. 

d.  Weekly  evaluation  worksheets  are  prepared  for  selected  sites. 

8.  EVALUATION  PROCEDURES.  The  QAE  will  visit  each  selected  3ite  on  the  day 
collection  is  scheduled.  Site  visits,  in  general,  should  be  conducted  four 
hours  after  scheduled  pick-up  time.  The  QAE  should  arrange  to  be  at  the 
site  at  the  time  of  pick-up  if  there  have  been  problems  reported  with  re¬ 
spect  to  debris  or  noise. 

9.  ANALYSIS  OF  RESULTS.  At  the  end  of  the  month,  the  QAE  is  to  count  the 
number  of  pick-ups  classified  a3  unsatisfactory  and  compute  an  overall  defect 
rate  for  the  month.  The  Observed  Defect  Rate  (ODR)  is  the  number  of  unsatis¬ 
factory  pick-ups  divided  by  the  sample  size. 


ODR 


(Total  ♦  U's) 
(Total  |  u's  +  S's) 


X  100 


Since  this  QA  Plan  is  based  on  random  sampling,  unsatisfactory  pick-ups 
detected  by  customer  complaints  cannot  be  used  in  computing  the  ODR.  Un¬ 
satisfactory  pick-ups  detected  by  customer  complaints  will  be  used  in  deter¬ 
mining  a  course  of  action  when  the  contractor's  observed  performance  is 
questionable  (i.e. ,  ODR  greater  than  the  AQL  but  less  than  l.S  times  the 
AQL). 


APPENDIX  D 


RANDOM  SAMPLING 


1.  GENERAL.  Random  sampling  is  a  surveillance  method  based  on  statistical 
theory.  The  key  element  o£  random  sampling  is  that  each  and  every  occurrence 
of  work  has  an  equal  chance  of  being  evaluated.  In  order  to  achieve  the 
desired  end  results,  knowledge  of  the  Contractor's  overall  performance  based 
on  evaluation  of  only  part  of  the  work,  surveillance  by  random  sampling  must 
be  applied  properly.  Key  elements  of  random  sampling  are: 

a.  Sample  size  is  specified  for  a  given  population  to  achieve  a  pre¬ 
determined  level  of  statistical  accuracy. 

b.  The  sample  of  work  occurrences  selected  for  evaluation  must  be  se¬ 
lected  by  a  random  process  in  which  each  occurrence  has  equal  chance  for 
selection. 

c.  Once  an  evaluation  schedule  has  established,  it  must  be  followed 
through  the  surveillance  period  (i.e.,  monthly  schedules). 

d.  Surveillance  data  gathered  by  other  methods  (i.e.,  customer  com¬ 
plaints  and  unscheduled  inspections)  can  not  be  combined  with  data  gathered 
by  random  sampling. 

e.  Assessment  of  the  Contractor's  overall  performance,  projected  from 
the  observed  condition  of  the  sample,  will  always  have  the  potential  to  be 
in  error.  Statements  as  to  overall  performance  should  be  3tated  as  "The 
Contractor's  overall  defect  rate  is  in  excess  of  X  %".  (Where  "X"  equals 
the  observed  defect  rate  minus  one  half  the  AQL  -  ex.  ODR  (9.2%)  -  1/2  AQL 
(10%)  -  4 . 2% . ) 

2.  MECHANICS  OF  RANDOM  SAMPLING.  Random  sampling  is  a  structured  approach 
based  on  statistics  to  contract  surveillance.  As  such,  there  is  a  set  pro¬ 
cedure  in  its  application.  The  mechanics  of  applying  random  sampling  are  as 
follows. 

3.  POPULATION.  The  total  number  of  work  occurrences  for  a  given  function 
that  are  to  be  performed  during  the  surveillance  period  must  be  known  ul 
accurately  estimated.  The  Inventory  of  Service  Worksheet  is  used  to  deter¬ 
mine  population  size. 

a.  When  work  is  scheduled,  population  size  i3  easy  to  determine. 

Example:  Activity  X  has  30  dumpsters,  70  are  emptied  weekly  and  10 
are  emptied  twice  a  week.  The  population,  total  number  of  work  occurrences 
per  month,  is  360  (70x4)  ♦  (10x4x2). 

b.  When  work  is  un  cheduled,  population  .ze  must  be  estimated. 

Example:  The  number  of  service  requests  for  the  past  six  months 

were: 
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Jan  321 
Feb  301 
Mac  295 
Apt  337 
May  340 
Jun  320 
Total  1914 

The  average  number  of  service  calls  per  month  has  been  319  (1914/6) .  This 
would  be  the  expected  population  for  service  calls  for  next  month  unless 
there  is  some  known  reason  to  expect  a  change. 

4.  SAMPLE  SIZE.  Sample  size  requirements  are  based  on  AQL,  population  size 
and  level  of  surveillance.  Sample  size  tables  are  used  to  determine  sample 
size.  Tables  for  normal  surveillance,  reduced  surveillance,  and  increased 
surveillance  are  attached. 

a.  Select  the  table  with  the  desired  level  of  surveillance  (Tables  I, 
II,  or  III). 

b.  Select  the  column  with  the  required  AQL  (.05,  .10,  .15,  .20,  or  .25) 

c.  Select  the  row  that  is  closest  to  the  population  size,  preferably 
the  next  largest  entry. 

d.  The  number  indicated  by  the  row  and  column  selection  i3  the  sample 
size  required  for  surveillance  in  one  surveillance  period. 

(1)  Daily  surveillance  requirements  will  be  determined  by  dividing 
the  required  period's  sample  size  by  the  number  of  days  that  surveillance  is 
to  be  conducted. 

(2)  Weekly  surveillance  requirements  will  be  determined  by  dividing 
the  required  sample  size  by  the  number  of  weeks  in  a  period. 

(3)  When  computing  weekly  or  daily  sample  sizes,  always  round  up  to 
the  next  whole  number  (e.g.  45  monthly  samples  required  and  20  work  days  per 
month  results  in  a  daily  sample  size  of  3  -  (45/20)  ■  2.25  and  rounded  up 
results  to  3) . 

5.  Sample  Selection.  The  final  thing  to  be  decided  in  sampling  is  how  the 
sample  will  be  drawn.  The  objective  in  the  method  is  to  insure  that  the 
sample  is  random  (that  is,  that  all  services  have  an  equal  chance  of  being 
selected) .  To  achieve  random  selection,  use  a  random  numbers  table  as  ex¬ 
plained  in  the  following  examples.  (A  random  numbers  table,  Table  IV,  is 
attached) .  Most  items  will  fall  into  one  of  these  examples. 

a.  Ose  Of  The  Random  Numbers  Table.  The  random  numbers  in  Table  IV  are 
arranged  in  groups  of  two. 

(1)  To  use  the  table,  begin  by  picking  at  a  random  a  group  of  num¬ 
bers  on  any  page  of  the  table.  This  is  usually  done  by  closing  the  eyes  and 
pointing  with  a  pencil  or  finger  to  some  initial  group. 


(2)  To  identify  additional  random  numbers,  follow  a  pattern.  Go 
along  a  given  line  to  its  end  and  then  along  the  next  line  to  its  end  and  so 
on  through  the  table  until  enough  numbers  have  been  selected  or  until  the 
table  ends. 

(3)  If  the  table  ends  and  there  are  still  more  numbers  to  select, 
go  back  to  the  beginning  of  the  table  and  continue  using  the  3ame  pattern. 

Ose  various  patterns  alternately,  for  example,  use  lines  for  one  sample,  use 
columns  for  the  next  sample,  and  use  a  diagonal  pattern  for  the  third  mple. 

b.  How  To  Ose  the  Random  Numbers  Table  To  Identify  a  Random  Sam  of 

Consecutively  Numbered  Work  orders.  Suppose  one  has  to  identify  a  a  * 

sample  of  97  work  orders  for  evaluation.  (Sample  size  is  based  on  a  ula- 

tion  of  319  using  normal  surveillance.)  This  can  be  done  at  the  beg  >g 

of  the  month  (before  the  work  orders  are  written)  or  at  the  end  of  t  It. 

(1)  If  there  are,  or  might  be,  319  consecutively  numbered  wot* 
orders  to  select  from,  then  one  begins  by  listing  the  lowest  work  order  nura- 
ber  (known  or  projected) .  This  could  be  1001,  or  possibly  443,  or  any  other 
sequentially  assigned  number.  List  the  highest  work  order  number  (known  or 
projected);  in  this  case,  it  could  be  #319  or  762.  Por  this  example,  use 
work  orders  numbered  1443  to  762. 

(a)  Select  97  three  digit  numbers  from  Table  IV  using  a  con¬ 
sistent  pattern. 

(b)  If  random  numbers  selected  are  not  between  443  and  762, 
discard  the  number  outside  the  designated  range  and  select  a  new  number. 

(2)  For  example,  using  the  initial  entry  on  Table  IV  we  would  select 
number  441.  This  number  is  too  low.  The  next  number,  going  down  the  column, 
is  343  again  too  low.  The  third  number  selected  is  749.  This  number  falls 
in  the  range  of  work  order  number  (443-762)  subject  to  inspection.  So  work 
order  number  749  is  selected  to  be  inspected.  The  next  work  order  number 
selected  is  523. 

(3)  This  process  would  be  continued  until  three  work  orders  are 
selected. 

c.  How  To  Ose  The  Random  Numbers  Table  To  Identify  A  Random  Sample  From 
A  Grouo  of  Items.  If  a  number  of  items  need  to  be  sampled  that  are  not  con¬ 
secutively  numbered,  the  simplest  solution  is  to  list  the  identifiers,  for 
all  the  items  in  a  column,  on  a  piece  of  lined  paper. 

(1)  Next,  number  the  lines  consecutively,  beginning  with  the  number 
one.  Now  use  the  random  number  table  to  draw  the  sample  from  the  line  num¬ 
bers.  A  selected  line  number  lead3  to  the  identifier  located  on  that  line, 
and  that  identifier  tell3  which  item  to  sample.  For  example,  if  one  chooses 
to  sample  a  set  of  work  orders  with  attached  sales  3lips,  one  is  not  going 
to  have  to  have  a  set  of  consecutively  numbered  work  orders  because  not  every 
work  order  has  a  sales  slip  attached. 
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(2)  List  the  work  orders  with  sales  slips  in  a  column,  number  each 
line  in  the  column,  and  randomly  select  enough  line  numbers  to  make  up  the 

sample. 

d.  How  To  Use  The  Random  Numbers  Table  To  Identify  a  Random  Sample  of 
Days.  Suppose  one  wants  to  identify  four  days  in  the  month  on  whicn  to  sam¬ 
ple  something.  The  days  of  the  month  can  be  numbered  01  to  31  (or  less,  as 
appropriate) . 

(1)  It  is  best  to  use  a  starting  point  different  from  the  one  used 
in  the  previous  example.  For  the  purpose  of  this  example,  it  is  being  used 
again. 


(2)  One  can  move  down  the  column  from  number  to  number  until  the 
first  number  between  01  and  31  is  spotted.  In  this  case,  it  is  22.  Thus 
the  22nd  day  of  the  month  is  selected  for  sampling. 

(3)  Continuing  in  this  fashion,  one  discovers  that  11  is  the  next 
number  selected.  This  number  is  disregarded.  Proceed  in  this  manner  until 
the  four  days  for  sampling  have  been  identified.  In  our  example,  the  4  days 
selected  would  be  22,  11,  10  and  24. 

(4)  If  it  is  not  desirable  to  sample  on  weekends,  discard  those 
days  selected  that  happen  to  fall  on  a  weekend  and  continue  that  selection 
until  the  proper  number  of  days  has  been  selected. 

e.  How  To  Use  the  Random  Mumbers  Table  Jo  Identify  a  Random  Sample  of 
Times  of  Day.  If  one  wants  to  select  random  times  of  day  to  sample  a  ser¬ 
vice  such  as  taxi  or  bus  service,  use  the  24  hour  clock. 

(1)  If  there  are  any  constraints  during  each  24-hour  period,  t3ke 
them  into  consideration.  For  example,  suppose  that  base  bus  service  opera¬ 
tes  between  0700  and  2300.  Convert  these  times  to  minutes  (e.g.  0700  =»  0, 
0410  »  130,  1215  ■  315,  etc.)  Again,  using  Table  IV  and  selecting  three 
digit  numbers  and  proceeding  across  the  line  from  the  initial  number,  one 
comes  to  441,  or  1421  hrs,  as  the  first  random  time. 

(2)  The  next  random  number  is  343,  or  1243  hrs.  The  number  is  good 
and  so  one  schedules  an  observation  for  1243  hrs. 

(3)  Proceed  in  this  manner  until  the  desired  number  of  sample  times 
have  been  identified. 

f.  How  To  Insure  Variety  in  the  Use  of  the  Random  Numbers  Table.  The 
use  of  variety  in  the  random  number  table  ensures  that  detectable  patterns 
do  not  occur. 

(1)  Success  In  using  the  tables  requires  consistency  but  also  vari¬ 
ety.  The  above  information  should  ensure  that  the  tables  are  properly  used 
and  that  the  sample  is  randomly  drawn. 

g.  Other  Random  Numbers  Generating  Methods.  The  use  of  a  hand  held 
calculator  with  a  random  number  generating  capability  is  an  alternative  to 
the  use  of  random  numbers  table.  Vsing  this  type  of  calculator  the  QAE  would 
enter  the  minimum  value  and  maximum  value  and  numbers  generated  would  always 
be  within  the  desired  range. 


TABLE  0-1 

SAMPLE  SIZE  REQUIREMENTS  FOR  REDUCED  SURVEILLANCE 
(SURVEILLANCE  LEVEL  I) 

POPULATION  AQL 

SIZE  .05  .10  _  .15  .20 


TABLE  D-2 

SAMPLE  SIZE  REQUIREMENTS  FOR  NORMAL  SURVEILLANCE 
(SURVEILLANCE  LEVEL  II) 


POPULATION  AQL 


SIZE 

.05 

.10 

.15 

.20 

.25 

50 

43 

37 

32 

27 

23 

75 

60 

49 

40 

32 

27 

100 

76 

58 

46 

37 

29 

125 

89 

66 

51 

40 

31 

150 

101 

72 

54 

42 

32 

175 

111 

78 

57 

43 

33 

200 

121 

82 

60 

45 

34 

225 

129 

86 

62 

46 

35 

250 

137 

89 

63 

47 

35 

275 

144 

92 

65 

48 

36 

300 

151 

95 

66 

48 

36 

325 

157 

97 

67 

49 

37 

350 

162 

99 

68 

49 

37 

,  3” 

167 

101 

69 

50 

37 

400 

172 

103 

70 

50 

37 

425 

176 

105 

71 

51 

38 

450 

181 

106 

71 

51 

38 

475 

184 

107 

72 

51 

38 

500 

188 

109 

72 

52 

38 

550 

195 

111 

73 

52 

38 

600 

201 

113 

74 

52 

39 

650 

206 

114 

75 

53 

39 

700 

211 

116 

75 

53 

39 

750 

215 

117 

76 

53 

39 

800 

219 

118 

76 

54 

39 

850 

222 

119 

77 

54 

39 

900 

226 

120 

77 

54 

39 

950 

229 

121 

78 

54 

39 

1000 

231 

122 

78 

54 

39 

1100 

236 

123 

78 

55 

40 

1200 

241 

124 

79 

55 

40 

1300 

244 

125 

79 

55 

40 

1400 

248 

126 

80 

55 

40 

1500 

251 

127 

80 

55 

40 

1600 

253 

128 

80 

55 

40 

1700 

256 

128 

80 

55 

40 

1800 

258 

129 

81 

56 

40 

1900 

260 

129 

81 

56 

40 

2000 

262 

130 

81 

56 

40 

2500 

269 

131 

32 

56 

40 

3000 

274 

132 

82 

56 

41 

3500 

277 

133 

82 

56 

41 

4000 

230 

134 

83 

57 

41 

4500 

282 

134 

33 

57 

41 

5000 

284 

135 

83 

57 

41 

6000 

287 

135 

83 

57 

41 

7000 

289 

136 

83 

57 

41 

8000 

290 

136 

84 

57 

41 

9000 

291 

136 

84 

57 

41 

10000 

292 

137 

84 

57 

41 
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&PPJNDIX  E 

EXTRAPOLATED  DEDUCTIONS  BASED  ON  SADDLING  TECHNIQUES 


Given: 

AQL  :  6.5*;  lot  size  :  450  units;  sample  :  50  units 
Tec  defectives  sere  found  in  the  sample. 

Kaximuir  contract  payment  per  month:  $10,000 
Payment  percentage  for  the  service  (from  PRS)  :  x  5* 
Kaximuir  payment  for  acceptable  service:  3500 
Unit  price  ($500  divided  by  450  units)  $1.11 
10  defectives  exceed  a  reject  number  of  9: 

Defective  percentage  in  sample:  20* 
Percentage  cf  sample  fcund  acceptable:  801- 
Credit  for  corrected  samples:  2.  2* 
Acceptable  percentage:  82.2* 
Payment  for  acceptable  service:  $411 


Figure  E. 1  Air  Force  Deductions  Under  Bandom  Sampling. 


This  figure  shows  how  deductions  would  normally  be  made 
when  a  contractor  exceeds  the  AQL  for  any  month's  perfor¬ 
mance,  This  example  illustrates  how  credits  would  be  applied 
when  work  is  allowed  to  be  reperformed.  Without  reperfor¬ 
mance,  the  above  final  payment  wou*d  instead  be  3400. 
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AQL  :  5*;  lot  size  :  50  units;  defectives  ;  5  units. 


Maximum  contract  payment  per  month:  *10,000 

Maximum  payment  percentage  for 

specific*  service:  x 

Maximum  payment  for  acceptable 

service:  3400 

Unit  price  ($400  divided  ty  50J  $8 

5  defectives  exceed  a  reject  number  of  3: 

Eercentage  cf  let  found  acceptable:  90% 

fayment  for  acceptable  performance:  3360 


Figure  B.2  Air  Force  Deductions  Based  on  Planned  Sampling. 


This  figure  illustrates  how  deductions  are  made  when 
planned  sampling  procedures  are  utilized.  Deductions  would 
not  be  taken  if  the  defectives  observed  were  less  than  the 
reject  number  of  three.  When  the  AQL  for  a  month's  perfor¬ 
mance  is  exceeded,  the  contractor  should  be  notified  by 
means  of  a  discrepancy  report.  If  successive  discrepancy 
reports  must  be  prepared,  a  Show  Cause  or  Cure  Notice  may  be 
required . 


All  items  are  subject  to  100%  inspection. 


Cost  of  required  services:  $5000 

Total  number  of  work  units(lot  size):  100 

Onit  price  ($5000  divided  by  100  units):  $50 

Sample  size:  100 

Observed  nonconforming  items:  6 

Units  reworked  at  government  option:  3 

Units  not  creditable  for  payment:  3 

Total  units  creditable  for  payment:  97 

Percentage  of  contract  price  due:  97% 

Cellar  value  due:  $4850 

Liquidated  damaces  at  10%  of 

nonconforming  service  (.10  c  6  x  $50):  $30 

Actual  amount  paid:  $4820 


Figure  E.3  NAVFAC  Payment  Under  100  Percent  Inspection. 


This  figure  illustrates  NAVFAC’s  method  of  deductions 
under  100$  inspection.  Liquidated  damages  are  assessed  at 
10%  of  the  value  of  initial  observed  defect  rates  in  the 
sample.  Items  may  be  resubmitted  for  credit  towards  final 
payment. 
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Fandom  sampling  is  being  utilized. 

All  defectives  are  reworked. 

Cost  of  required  services:  $5000 

Total  work  units  (lot  size)  :  1000 

Unit  price  ($5000  divided  by  1000  units)  $5 

Sample  size:  122 

Observed  nonconfe  aing:  11 

Percentage  nonconforming:  S’ 

AQL :  1 0X 

Units  reworked  at  government  option:  11 

Total  units  creditable  for  payment:  1000 

Percentage  of  contract  price  due:  100% 

Cellar  value  due:  $5000 

liquidated  damaaes  at  10%  of 

nonconforming 'service  (.10  x  11  x  $5):  $5.5 

Actual  amount  paid:  $4994.5 


Figure  E.4  SAVFic  Fandom  Sampling  Deductions  Bhere  ACL  is 
Not  Exceeded. 


Under  this  scheme,  random  sampling  procedures  are  being 
utilized.  As  before,  liquidated  damages  are  assessed  against 
all  ncnccnfcrming  items  that  are  observed.  If  all  items  are 
reworked,  as  in  this  case, 
damages  is  made. 


full  payment  less  any  liquidated 


Random  sampling  procedures  are  being  utilized. 


Cost  cf  required  servics  $5000 

Total  number  of  work  units  (lot  size)  :  1000 

Unit  price  ($5000  divided  by  1000)  $5 

Sample  size:  122 

Observed  nonconforming  units:  11 

Threshold  ( AQL)  :  10'. 

Observed  percentage  nonconforming:  91 

Units  reworked  (at  Government  option)  :  6 

Units  net  creditable  for  payment:  5 

Total  units  creditable  for  payment:  995 

Eercentage  cf  contract  price  due:  99.515 

Dollar  value  due:  $4975 

liquidated  damaces  (101  of  nonconforming 

items,  or  .10'x  1i  x  $5)  $5.5 

Actual  amount  paid:  $4969.5 


I 

I 


I 

I 


Figure  E.5  NAFFAC  Deductions  when  defectives  are  less  than 
AQL  and  sone  are  reworked. 


This  figure  shows  that  random  sampling  is  the  inspec¬ 
tion,  and  that  not  all  of  the  units  found  defective  were 
reworked.  Thus,  deductions  for  liquidated  damages  and  defec¬ 
tive  observed  sample  items  are  ta*en.  Ians  procedure  is  not 
utilized  by  the  Air 


Force 


W 


r 
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Bandcm  sampling  procedures  ara  being  utilized. 


Some  defectives  are  reworked  in  the  allotted  time. 

Cost  of  required  services:  $ 5000 

Total  number  of  work  units  (lor  size):  1000 

Onit  price  ($5000  divided  by  1000  units):  $5 

Sample  size:  122 

Observed  defectives  nonconforming:  17 

Percentage  nonconforming:  1  '-i% 

Threshold  (A QL)  :  103 

Extrapolation  of  defective  percentage 

to  obtain  total  acceptable  units:  860 

Units  reworked  (at  Government  option)  :  14 

Total  units  creditable  for  payment:  874 

Percentage  cf  contract  price  due:  87.43 

Cellar  value  due:  $4390 

liquidated  damaces  at  103  of  the  value 
of  nonconforming  services  based  on 
extrapolated  percentages  (.14  x  1000  x  5  x  .  1):$70 

Actual  amount  paid:  $4320 


Figure  B.6  MAT FAC  Deductions  where  AQL  is  exceeded  and 
some  defectives  are  reworked. 


This  illustration  shows  that  when  random  sampling 
reveals  observed  defect  rates  in  the  sample  to  be  greater 
than  the  specified  AQL ,  liquidated  damages  and  payment 
deductions  use  sample  defect  percentage.  Units  that  are 
resubmitted  are  credited  only  after  the  extrapolation  calcu¬ 
lations  are  complete.  This  procedure,  in  concert  with  reli¬ 
able  statistical  techniques,  should  be  most  effective  in 
gaining  contractor  attention  to  defective  performance. 
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Planned  sampling  inspection  is  in  effect 


Cost  of  required  service: 

Tonal  number  of  work  units  (lot  size)  : 

Unit  price  ($5000  divided  by  1000)  : 

Sample  size  (as  desired): 

Observed  nonconforming  units: 

Units  reworked  within  allotted  time: 

Total  units  creditable  for  payment: 

Percentage  of  ccntract  price  due: 
(Deduct  only  for  observed  defects) 

Cellar  value  due: 

Liquidated  damages  at  10%  of  value  of 
nonconforming  services: 

Actual  amount  paid: 


$5000 

1000 


99.  U% 


$4970 


$4964 


Figure  B.7  NAT FAC  Deductions  under  planned  saapling  proce¬ 

dures. 


The  figure  shows  the  deduction  procedures  for  planned 
sampling. 


APPENDIX  P 

INSPECTION  BESOOBCE  ESTI NATION  WORKSHEETS  DEVELOPED  BT 

LANTDIV 

The  following  three  worksheets  are  examples  of  QAE 
resource  estimating  worksheets  that  have  teen  developed  and 
promulgated  by  the  Atlantic  Division,  Naval  Facilities 
Engineering  Command.  These  worksheets  apply  for  planned, 
randcir  and  100  percent  inspection  methods  and  should  result 
in  fairly  accurate  estimates  of  QAE  requirements.  Ail 
monthly  totals  can  be  accumulated  to  obtain  annual  hour 
reguirement s ,  and  thus  provide  an  estimate  of  personnel 
ceiling  points  that  should  either  be  retained  or  requested. 
This  methodology  would  provide  future  benefits  in  that  agen¬ 
cies  might  te  able  tc  accumulate  actual  hours  expended  for 
inspection  during  the  performance  period  of  CA  contracts, 
and  thus  result  in  a  manhour  estimating  factor  for  various 
work  requirements  that  are  performed  under  the  contract. 

Items  listed  in  the  contract  requirements  columns  should 
correspond  tc  work  items  that  are  listed  on  the  performance 
requirements  summaries  (PBS's). 


215 


LANTNAVFACENCCOMINST  11014. 4D 

QAE  WORKLOAD  CALCULATIONS  PLANNED  SAMPLING 


INSTRUCTIONS:  Fill  in  Che  contract  requirement,  quantity  of  work  and  level  of 
surveillance  from  Performance  Requirement  Summary  Table. 
Calculate  Che  number  of  inspections.  Using  the  estimated  hours 
per  inspection,  calculate  the  number  of  hours  of  inspection 
required  per  month.  Refer  to  MO-327  for  suggested  surveillance 
levels. 


- 

Quantity 

Level  of 

Number  of 

Hrs.  per 

Monthly 
Hrs.  of 

Contract  Requirement 

of  Work 

Surveillance 

InsDec . 

Inspecs. 

LANTNAVFACENGCOMINST  11014. 4D 


QAE  WORKLOAD  CALCULATIONS  1002  INSPECTION 


INSTRUCTIONS:  Fill  in  Che  contract  requirement  and  quantity  of  work  fro® 
Performance  Requirement  Summary  Table.  The  level  of 
surveillance  is  1002.  The  quantity  of  work  is  the  number  of 
inspections.  Using  the  estimated  hours  per  inspection, 
calculate  Che  number  of  hours  of  inspection  required  per  month. 


Quantity  Level  of  Number  of  H 

Contract  Reauirement  I  of  Work  Surveillance  Insoecs.  I 


Monthly 
Hrs.  of 
Inspecs. 


1002 


LANTNAVFACEWCCOMINST  11014. 4D 


QAE  WORKLOAD  CALCULATIONS  RANDOM  SAMPLING 


INSTRUCTIONS:  Fill  in  the  contract  requirement,  quantity  o£  work  and  the 
number  of  inspections  to  be  performed  from  Performance 
Requirement  Summary  Table.  Indicate  surveillance  level.  Using 
the  estimated  hours  per  inspection,  calculate  the  number  of 
hours  of  inspection  required  per  month.  Refer  to  MO-327  for 
recommended  surveillance  levels  and  associated  sample  size 
requirements. 


Contract  Reauirement 


Monthly 

Quantity  Level  of  Number  of  Hrs.  per  Hrs.  of 

of  Work  Surveillance  Inspecs.  Inspec.  Insoecs. 
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VABICOS  Cl  CONTRACT  ADMINISTRATION  ORGANIZATIONS  STUDIED  ET 
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PWC  =  public  Works  Officer 

AEWO  =  Assistant  Public  Works  Officer 

ESC  Office  =  Facilities  Service  Contract  Office 

MCE  =  Maintenance  Control  Division 

SC M  =  Service  Contract  Manager 

EE'S  =  Planners  and  Estimators 

CAE's  =  Quality  Assurance  Evaluators 

This  oroanization  locates  the  FSC  office  as  a 
completely  centralized  activity  division. 


Figure  G.1  Centralized  Organization  With  SCM  as  Manager. 
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APWO 


Functional  MGS 


HOUSG 


SECTY 


TRANS 


ICAE'SI  IQAE'SI  IQAE'SI  IQAE'SI 


Identical  tc  Figure  1;  adait lonally: 

HOUSG  =  Housing  Division 
SECTY  =  security  Division 
TRANS  =  Transportation  Division 

SCM's  supervise  a  Halted  number  of  (or  no) 
CAE's.  QAE's  primarily  work  for  functional 
managers.  SCM  functions  more  as  an  advisor. 


Figure  G.  3  Decentralized  Systea  with  sen  in  mcd. 
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]  PWO  1 


Leqgnd 

Sane  as  in  Figures  1  and  3 

In  this  organizational  format,  the  SCM  is  a 
division  head  of  the  activity.  Direct  control 
of  QAE*  s  is  exercised  by  functional  manaaers. 
The  SCM  acts  primarily  m  an  advisory  capacity 


APPENDIX  9 

AFLC  SOPELEHENTABI  POLICY  FOR  QAE  BRANCH  ORGANIZATION 


Id  this  Appendix,  AFLC  Supplement  No.  1  is  provided.  It 
illustrates  amplifying  Air  Force  Logistics  Command  Guidance 
which  pertains  to  AFF  70-9. 


223 


DEPARTMENT  OF  THE  AIR  FORCE 
Headquarters  Air  Force  Logistics  Command 
Wnght-Paeterson  Air  Force  Base  OH  45433 


AFLC SUPPLEMENT  1 
AKR  TO-9 
26  February  1982 


Contracting  and  Acquisition 
BASE  LEVEL  CONTRACT  ADMINISTRATION 


AFR  70-9.  25  September  1979.  is  supplemented  as  follows: 

lc.  Where  separate  contract  administration  offices  exist,  the 
term  "BCO"  in  the  basic  regulation  is  interpreted  as  "ACO." 
except  in  paragraphs  lc  and  8d. 

6a.  AFLC/PMM  is  OPR  for  the  QAE  program  within 
AFLC. 

6b.  All  QAEs  will  attend  a  phase  I  QAE  training  program, 
managed  by  the  local  QAE  program  coordinator.  JO  days 
before  the  date  of  initial  contract  performance.  If  the  train¬ 
ing  can't  be  completed  :»  days  before  contract  start,  the 
contract  file  will  contain  the  documented  reason  why  the 
training  couldn't  be  done  within  the  specified  time.  Send  a 
copy  of  the  documentation  to  the  functional  area  chief  (FAC). 
Die  training  program  will  be  divided  into  two  phases:  The 
administrative  contracting  officer  (ACO)  is  responsible  for 
phase  I  training,  which  is  basically  an  orientation.  It  covers 
provisions  of  individual  contracts  over  which  the  QAE  will 
have  surveillance  responsibility.  Phase  II  training  will  be 
conducted  using  information  and  materials  supplied  by  HQ 
USAF/RliCL.  and  will  be  completed  at  the  first  quarterly 
trainingsession  ofter  appointment.  This  training  is  applicable 
to  all  contracts.  The  QAE  program  coordinator  will  give  the 
Civilian  Personnel  Office  a  list  of  names  of  all  the  people  who 
have  sucessfull  completed  the  Air  Force-prepared  training 
course.  The  Civilian  Personnel  Training  Office  will; 

(1)  Authenticate  completion  of  the  training  on  DD 
Form  1556.  Request.  Authorization.  Agreement.  Certification 
of  Training  and  Reimbursement. 

(2)  Place  a  copy  of  DD  Form  1556  in  the  individual’s 
official  personnel  folder. 

(3)  Give  a  copy  to  the  individual,  the  QAE  program 
coordinator,  and  the  cognizant  FAC. 

(4)  Keep  records  on  file  for  persons  completing  this 
training. 


7b.  Upon  receipt  of  Letters  of  Nomination  from  the  FAC.  the 
commander  » ill  appoint  QAEs  and  their  alternates  in  writing. 
A  suffiaait  number  of  QAEs  will  be  appointed  to  make  sure 
adequate  surveillance  is  provided  at  all  times. 

8d.  Give  copies  of  the  letter  of  appointment  to  the 
contracting  officer,  along  with  the  purchase  request  for 
recurring  contract  requirements.  If  a  cost  study  is  being 
conducted,  furnish  the  letter  of  ap|«untment  as  soon  as  the 
QAE  is  selected.  The  FAC  will  send  a  copy  of  the  letter  to  the 
QAE  program  coordinator  and  the  saw  icing  Civilian 
Personnel  Office.  The  Civilian  Personnel  Office  will  make  sure 
AF  Form  1278.  Civilian  Personnel  Position  Description, 
reflects  duties  of  emplovees  named  as  the  QAE  and  alternate 
QAE. 

8j.  Develop  surveillance  plans  (SP)  for  those  contracts  that 
don’t  have  standard  Performance  Work  Statements  (PWS)  to 
ensure  proper  contract  surveillance.  The  contracting  officer. 
FAC.  and  QAE  should  jointly  prepare  SPs  and  revisions  to 
them  and  should  tailor  them  to  the  individual  contract.  The 
frequency  of  QAE  surveillance  and  the  type  of  inspection  (for 
example,  random  sampling.  100  percent  inspection, 
surveillance  check-list)  for  each  surveillance  area  should  be 
specified.  Do  this  before  the  contract  startdate.  If  experience 
with  the  contractor  shows  that  extensive  surveillance  isn’t 
required,  the  SP  may  be  revised  to  reduce  the  level  of 
frequency  of  surveillance.  Any  revisions  to  SP  require  the 
concurrence  of  the  contracting  officer.  Where  SPs  take  the 
form  of  checklists,  attach  them  to  the  AF  Form  372.  Contract 
Monitoring  and  Surveillamce  Report,  and  submit  to  the 
contracting  officer  according  to  paragraph  10c. 

9d-  The  ACO  with  the  help  of  the  QAF7  program  coordinator, 
will: 

(1)  Give  guidance  to  the  FAC  and  QAE  as  appropriate 
regarding  contract  interpretation,  resolution  of  problems, 
requirements  of  this  regulation,  and  other  related  contracting 
matters 


Supersedes  AFR  70-9/AFLC  Sup  1.  5  May  1978. 

No  of  printed  pages:  3 
OPR:  PMM  (W.  Ely) 

Approved  hy:  Col  Paul  Baldasari 
Writer-Editor:  C.  Rainey 

Distribution.  F.X  (HQ  L’SAF/LGP:  HQ  AF1SC/DAP:  AUL/SE . 1  ea> 


2 


AFR  70-9/ A  FLC  Sap  1  2fi  February  1982 


(2)  Indoctrinate  the  QAE  and  alternate  QAE  as  to  the 
extent  of  authority,  responsibility,  and  limitation,  as  outlined 
in  the  contract  and  this  regulation.  Do  this  prior  to 
contractor  performance. 

9e.  Prwidecopies  of  contracts  and  contract  modifications  to 
the  cognizant  QAE.  as  well  as  the  FAC.  prior  to  service  start 

dates. 

If  the  AFO  and  FAC  agree  that  a  quarterly  meeting  isn't 
required,  document  the  contract  file.  When  quarterly 
meetings  are  held.  |iersonnel  attending  will  include  ;is  a 
minimum  the  contracting  officer.  QAE  program  coordinator. 

the  FAC.  and  the  QAE. 

9h.  Furnish  the  QAE  with  a  information  copy  of 
correspondence  iiertaining  to  contract  requirements  or 
discrepancies  sent  to  or  received  from  the  contractor. 

9k(Addedli  The  local  QAE  program  coordinator  is  the  focal 
point  far  the  QAE  program.  The  QAE  program  coordinator 
will  keep  a  record  of  ail  contracts  requiring  QAEs  and  the 
names  and  training  records  of  QAEs  aopointed.  The  QAE 
program  coordinator  may  be  required  periodically  to  provide 
■■formation  on  the  QAE  program  to  HQ  AFLC/PMM.  Any 
reporting  requirements  developed  in  support  of  the  QAE 
program  wall  comply  with  AFK  178-7.  as  supplemented.  The 
QAE  program  coordinator  will  work  closely  with  cognizant 
contracting  officers  in  making  sure  QAEs  understand  the 
responsibilities  and  the  surveillance  of  specific  contracts.  The 
QAE  program  coordinator  will  make  sure  records  are 
maintained  of  the  conferences  held  accordingto  paragraph  9g. 

10b.  The  Chief  QAE  is  responsible  for  certification  of 
contractor  invoices  that  services  were  satisfactory. 

10«.  The  QAE  will  maintain  appropriate  forms  <AF  Form 
802.  Contract  Discrepancy  Report:  AF  Form  799.  Surveillance 
Activity  Checklist)  for  each  contract.  The  use  of  AFL.C  Form 
134.  TRCO  Daily  Log.  is  authorized  only  until  the  present 
supply  of  the  form  is  exhausted,  or  until  new  contracts  go  into 
effect.  AFLC  Form  134  is  not  authorized  for  contracts 
awarded  after  date  of  this  publication.  Type  entries  or  record 
them  legibly  in  ink: 

(1)  Enter  and  sign  on  the  forms,  as  required  by  the 
contracting  officer  for  services  iierformed.  a  notation  of  the 
contractor's  performance  including  any  deficiencies.  For 
services  iwrformed  on  other  than  a  daily  basis,  enter  and  sign 
the  notation  when  the  contractor  actually  performs  the 
services.  Annotate  this  only  when  the  QAE  performs  an 
nspection. 

(2)  Record  any  performance  deficiencies  noted  in 
precise,  descriptive  language.  Notify  the  contractor's  project 
manager  tor  authorized  representative)  and  request 
sdenow  ltdgement  by  concurrence  or  nonconcurrence  in  the 


Remarks  section  of  the  forms  that  thedeficiency  does  exist  If 
the  contractor  concurs,  a  statement  is  required  outlining 
actions  planned  or  taken  to  correct  thedeficiency  and  prevent 
its  recurrence.  In  cases  of  contractor  nonconcurrence.  the 
QAE  should  immediately  contact  thecontracting  officer  If  the 
contractor's  project  manager  (or  authorized  representative) 
isn't  aviuiahle  to  discuss  the  discrepancies,  notify  the 
contracting  officer  who.  in  turn,  notifies  the  contractor  (by 
letter  if  time  permits)  of  the  deficiencies.  When  prompt 
corrective  action  isn't  taken  or  when  a  deficiency  becomes 
more  serious,  notify  the  contracting  officer  ami  the  FAC. 
Document  the  notification  by  annotating  the  AF  Form  802. 


(3)  Hand-carry,  when  feasible,  the  QAE  reports  for  the 
previous  month  (assembled  and  sequentially  numbered)  to  the 
contracting  officer  by  the  5th  workday  of  each  month,  or  more 
often  if  deemed  necessary  by  the  contracting  officer.  The 
contractingofficerwil!  review  and  initial  theQAE  reports  and 
will  then  place  the  reports  in  the  official  contract  file  unless 
the  contracting  officer  determines  that  the  IJAE  should  keep 
the  reports.  In  that  case,  the  reports  are  furnished  to  the 
contracting  officer  for  inclusion  in  the  contract  file  upon 
contract  completion. 

lOe(Added).  Limitations  of  Authority.  QAE  personnel  will 


(1)  Clarify,  make,  or  infer  legal  interpretations  on  the 
scope  or  intent  of  the  contract. 

(2)  Approve  contractor  procedures  unless  specifically 
provided  by  the  terms  and  conditions  of  the  contract.  When 
contractually  authorized,  approval  authority  remains  subject 
to  any  limitations  the  contracting  officer  may  impose. 

(3)  Authorize  expenditure  of  funds,  except  under  the 
specific  terms  of  the  contract. 

(4)  Levy  or  impose  upon  contractors  any  task  or  permit 
any  substitution  not  specifically  provided  for  in  the  contract 

(5)  Enter  into  contractual  agreements  including 
contract  modifications. 

(fi)  Give  direction  to  the  contractor  or  to  employees  of 
the  contractor. 

(7)  Offer  advice  to  the  contractor  which  may  adversely 
affect  contract  performance,  compromise  the  rights  of  the 
Government,  provide  the  basis  of  a  claim  for  constructive 
change,  or  impart  any  pending  or  future  contracting  officer 
determinations  as  to  fault  or  negligence. 

1 1.  Waiverstotherequircmenlsofthisparagraphwill 
be  in  writing,  signed  by  the  FAC.  Send  copies  to  the  QAE 
program  coordinator  and  the  contracting  officer. 
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12.  Send  all  requests  for  deviations  to  HQ  AFLC/PMM. 

OFFICIAL 

WILLIAM  R.  CARROLL  Colonel.  USAF 
Director  of  Administration 


JAMES  P.  MULLINS.  General.  USAF 
Commander 


APPENDIX  I 

ABBREVIATION S  OSED  IN  THE  STUDY  TEXT 

Abbreviation  Description 


A-76 

AFR 

AFSCAG 

AFLC 

A  £L 

ASBCA 

BCB 

BOS 

CA 

CNM 

C  NO 

DOD 

EFD 

FY 

GAO 

GS 

LANTCIV 

MOD 

MEO 

MIL-STD-105D 

NAVFAC 

NAV5DE 

NCRTHDIV 

NBCC 

NSC 

0  F?  P 

CBS 


0 ME  circular  A-76 
Air  Force  Regulation 

Air  Force  Service  Contract  Advisory  Group 

Air  Force  Logistics  Command 

Acceptable  quality  level 

Armed  Services  Board  of  Contract  Appeals 

Bureau  of  the  Budget 

Base  Operating  Support 

Ccmmercial  Activities 

Chief  of  Navy  Material 

Chief  of  Navy  Operations 

Department  of  Defense 

Engineering  Field  Division 

Fiscal  Year 

General  Accounting  Office 

General  Schedule  Series 

Atlantic  Division,  NAVFAC 

Maintenance  Control  Division 

Most  Efficient  Organization 

Military  Standard  1050 

Naval  Facilities  Engineering  Command 

Naval  Supply  Systems  Command 

Northern  Division,  NAVFAC 

Navy  Regional  Contracting  Center 

Naval  Supply  Center 
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